-- Logs begin at Fri 2022-10-21 14:13:55 UTC, end at Fri 2022-10-21 15:25:37 UTC. -- Oct 21 14:44:22.657162 dv-cinder-836083-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Oct 21 14:44:22 2022] *** Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: nodename: dv-cinder-836083-16-iscsi Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: machine: x86_64 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: clock source: unix Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: pcre jit disabled Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: detected number of CPU cores: 4 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: current working directory: / Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: detected binary path: /usr/bin/uwsgi-core Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: your processes number limit is 47680 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: your memory page size is 4096 bytes Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: detected max file descriptor number: 2048 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: lock engine: pthread robust mutexes Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: thunder lock: enabled Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 21 14:44:22.673981 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 21 14:44:22.708291 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: Python main interpreter initialized at 0x55d31946e9c0 Oct 21 14:44:22.708291 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: python threads support enabled Oct 21 14:44:22.708291 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: your server socket listen backlog is limited to 100 connections Oct 21 14:44:22.708291 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: your mercy for graceful operations on workers is 90 seconds Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: mapped 403077 bytes (393 KB) for 2 cores Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: *** Operational MODE: preforking *** Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: *** uWSGI is running in multiple interpreter mode *** Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: spawned uWSGI master process (pid: 81551) Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: spawned uWSGI worker 1 (pid: 81564, cores: 1) Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: spawned uWSGI worker 2 (pid: 81565, cores: 1) Oct 21 14:44:22.714617 dv-cinder-836083-16-iscsi devstack@keystone.service[81551]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 21 14:44:22.709184 dv-cinder-836083-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 21 14:44:27.050334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 14:44:27.050334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: from cryptography.utils import int_from_bytes Oct 21 14:44:27.051110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 14:44:27.051110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: from cryptography.utils import int_from_bytes Oct 21 14:44:27.051554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 14:44:27.051554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: from cryptography.utils import int_from_bytes Oct 21 14:44:27.051685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 21 14:44:27.051685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: from cryptography.utils import int_from_bytes Oct 21 14:44:27.101195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 21 14:44:27.101387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 21 14:44:27.101536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 21 14:44:27.101630 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] command line args: [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 21 14:44:27.101707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 21 14:44:27.101872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] command line args: [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 21 14:44:27.101956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 21 14:44:27.102110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ================================================================================ {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 21 14:44:27.102194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 21 14:44:27.102306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ================================================================================ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 21 14:44:27.102390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] admin_token = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.102561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] admin_token = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.102668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] config_dir = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.102790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.102873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_dir = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.102971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] config_source = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] debug = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] config_source = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] debug = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.103979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_config_append = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_config_append = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.104950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_dir = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_dir = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_file = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.105915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.106964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.107957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.108928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.60/identity {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] publish_errors = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] publish_errors = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.109962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.110898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] standard_threads = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] standard_threads = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] use_journal = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] use_json = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_journal = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.111975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] use_stderr = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_json = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] use_syslog = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_stderr = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] use_syslog = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 21 14:44:27.112691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.112853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.112927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.113892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.114877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115757 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.115983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.116963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.117938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.118770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.123835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.123949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.124934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.125995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.126967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.127986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.128940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.129888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.130932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.131921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.132958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.133977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.134929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.135985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.136988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.external = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.137895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.external = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.password = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] auth.token = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.138930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.password = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] auth.token = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.139967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140668 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.140911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.141071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.141635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.141905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.142943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.143901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.144928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.145990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.146923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.147881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.148878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.149955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.150967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.151921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.152915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.153857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.154860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.155997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.156726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.157476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.157566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.157697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.157775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.157845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.158889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.159999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.160958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.161995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.162977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.163910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.164848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.165985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.166953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.167921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.168945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.169896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.170859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.171992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172585 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] role.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.172903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] role.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.173878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.174898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.176931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.177872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.178962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.179924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.180994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.181924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.182941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.183878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.184857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.185849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.connection = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.186860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.187953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.188921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.189909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81564) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 21 14:44:27.190861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 21 14:44:27.190965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81565) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 21 14:44:27.229887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.230028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.230108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.230195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.230270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.230346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.243431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.243720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.276406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.276641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.277910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.278163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.334246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.334494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.383310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.383547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.384525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.384767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.548934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.549007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.549080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.549080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.569778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.570931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.571007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.571108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.571191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.571350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.587123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.587378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.599970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.600898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81565) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.601693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81564) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 21 14:44:27.617961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.626166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 14:44:27.626166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.626427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.626557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 14:44:27.626969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 14:44:27.627122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.627269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.627399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.627529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 21 14:44:27.627657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.627786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.627923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.628051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.628051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.628051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.630176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.630464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.630782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.630923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.631055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.631184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.631327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.631595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.638020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.638560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.640960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.642006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.642163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.642344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.642474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.642660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.642792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.642933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.643918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.644757 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.645035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.667433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.671143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.671234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.671234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.671361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.671361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.673845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.674136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.674428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.676064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.676351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.676635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.680726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.681021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.683058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.683454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.686724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.687027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.687338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.687775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.687875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.687952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.688529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.688787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.688881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.689922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.690200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.705575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.705858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.706641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.706931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.709733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.710909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.712972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.713276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.714110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.714411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.717467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.718604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.723448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.723539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.723994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.724301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.724594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.724887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.725151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.725234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.725454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.725536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.725740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.726038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.726329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.726622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.741615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.741910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.742185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.742455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.742539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.742819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.743090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.743356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.749341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.750238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.751571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.752493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.753131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.753423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.754154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.754446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.756171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.756459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.756759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.757051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.757132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.763535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.763670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.763752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.763752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.763752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.769393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.769683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.769981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.770067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.770269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.770354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.770689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.770820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.770982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.771065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.771535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.771632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.771853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.772137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.786906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.787751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.790155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.790677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.792824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.793674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.794256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.795142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.795762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.796059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.796463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.796742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.797036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.797321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.803504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.804472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.804925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.805943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.807847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.808241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.808532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.808815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.809085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.809376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.809661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.809937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.817188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.817480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.821064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.821151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.821151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.821151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.824681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81565) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.824968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.825255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.825539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.825822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.826102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.826382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.826715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.826807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81564) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 21 14:44:27.826979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.827965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.828158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.828248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.828434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.828516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81565) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.828774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.829076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.829347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.829628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.829899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.830164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.830435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81564) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 21 14:44:27.863592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d31946e9c0 pid: 81565 (default app) Oct 21 14:44:27.864964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d31946e9c0 pid: 81564 (default app) Oct 21 14:44:27.866956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac4d2-476e-4295-92ea-5a08c196b633 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:27.867249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac4d2-476e-4295-92ea-5a08c196b633 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:27.867563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac4d2-476e-4295-92ea-5a08c196b633 None None] PATH_INFO: `/v3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:27.871762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1/1] 10.222.0.60 () {50 vars in 791 bytes} [Fri Oct 21 14:44:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:35.551876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:35.552392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:35.552474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:35.556722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1/2] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:44:35.562100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:35.562525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:35.562666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:35.633724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: INFO dbcounter [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] Registered counter for database keystone Oct 21 14:44:35.652683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG oslo_db.sqlalchemy.engines [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81565) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 21 14:44:35.659466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writer thread running {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 21 14:44:35.839486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG passlib.handlers.bcrypt [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81565) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 21 14:44:35.839802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG passlib.handlers.bcrypt [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81565) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 21 14:44:35.898425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:35.900157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:36.029095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 2/3] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 21 14:44:36.494744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:36.564572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: INFO dbcounter [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] Registered counter for database keystone Oct 21 14:44:36.585671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG oslo_db.sqlalchemy.engines [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81564) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 21 14:44:36.591761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writer thread running {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 21 14:44:36.709133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf3b5c-0c19-4621-9b0a-763f6c28a181 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:36.713429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:36.714051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:36.714329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:36.714603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:36.723462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:36.775951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING keystone.common.rbac_enforcer.enforcer [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 21 14:44:36.776305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0ac74-184b-4f84-b23f-4dcbecb3185e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:36.785635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 2/4] 10.222.0.60 () {58 vars in 1324 bytes} [Fri Oct 21 14:44:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 14:44:36.791599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:36.808619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2113989c-38e7-4779-901b-309a33d260e6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:36.813032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:36.814138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:36.814420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:36.814727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] PATH_INFO: `/v3/projects/1cd93ea5b05642848d07178b50fbac9f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:36.816928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] RBAC: Authorizing `identity:get_project(project_id=1cd93ea5b05642848d07178b50fbac9f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:36.869536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 21 14:44:36.869879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56d5238-2efc-49e6-8b82-532b3b123941 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:36.874373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 3/5] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:44:36 2022] GET /identity/v3/projects/1cd93ea5b05642848d07178b50fbac9f => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:38.990697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481666-033d-4804-8e04-e148710e7c2b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:38.991081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481666-033d-4804-8e04-e148710e7c2b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:38.991256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58481666-033d-4804-8e04-e148710e7c2b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:38.995350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 3/6] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:44:39.000781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:39.001208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:39.001340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:39.022412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:39.023947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:39.031798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 4/7] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 14:44:39.303238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58481666-033d-4804-8e04-e148710e7c2b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:39.315431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58481666-033d-4804-8e04-e148710e7c2b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:39.319685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:39.320281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:39.320562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:39.320835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:39.328415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:39.330181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f7589-3595-48b9-937a-35711acf0f8d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:39.339455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 4/8] 10.222.0.60 () {58 vars in 1324 bytes} [Fri Oct 21 14:44:39 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 14:44:39.344754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:39.354538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a94dac-e9d3-4eac-9373-a81d24aa03fb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:39.358863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:39.359486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:39.359770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:39.360043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] PATH_INFO: `/v3/users/332151c44437481cb5ef58e0902276ac` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:39.361343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] RBAC: Authorizing `identity:get_user(user_id=332151c44437481cb5ef58e0902276ac)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:39.363214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7bd36a-93a1-4e66-9b66-efb495e15467 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:39.364726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 5/9] 10.222.0.60 () {56 vars in 1198 bytes} [Fri Oct 21 14:44:39 2022] GET /identity/v3/users/332151c44437481cb5ef58e0902276ac => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:41.552043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5550d-6654-45fb-b9e0-7e600154455b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:41.552559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5550d-6654-45fb-b9e0-7e600154455b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:41.552559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5550d-6654-45fb-b9e0-7e600154455b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:41.553885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 5/10] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:44:41.559014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:41.559449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:41.559690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:41.578417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:41.579621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:41.600758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74424f-a70a-4c53-a209-c6d82c6bf4cf None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:41.601044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74424f-a70a-4c53-a209-c6d82c6bf4cf None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:41.601313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b74424f-a70a-4c53-a209-c6d82c6bf4cf None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:41.602624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 6/11] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:44:41.607729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:41.608014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:41.608289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:41.651549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG passlib.handlers.bcrypt [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81564) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 21 14:44:41.651943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG passlib.handlers.bcrypt [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81564) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 21 14:44:41.661627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 6/12] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:41.702396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:41.704019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:41.813272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 7/13] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:44:41.927442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:41.954044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.036081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d902bc8-3939-4527-a5c6-fc4616161b79 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.070085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.070685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.070966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.071238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.079263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.084009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:42.092433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:42.093564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:44:42.094704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 7/14] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:44:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:44:42.099915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.180311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.182578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745051a7-ecaa-4899-b8b6-83b89959add4 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.184904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.198752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7794da5-4afc-4feb-9db2-cbfe73c9aa50 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.218582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.219481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.219766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.220040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.221622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.223305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.233190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.233797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.234073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.234342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.235049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:42.235354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.236895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.248503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 8/15] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:42.254402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.276899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.sql.core [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:44:42.276899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:44:42.276899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [parameters: {'id': '70f2b5873261494e94beee335c99c8d4', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 14:44:42.276899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81564) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:44:42.296409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application result.read() Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 14:44:42.297235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application yield resource Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise self.value Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application session.commit() Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 14:44:42.298144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self.session.flush() Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application compat.raise_( Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 14:44:42.299604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 14:44:42.300820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application result.read() Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:44:42.301560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [parameters: {'id': '70f2b5873261494e94beee335c99c8d4', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 14:44:42.302362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:42.303146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 8/16] 10.222.0.60 () {60 vars in 1137 bytes} [Fri Oct 21 14:44:42 2022] POST /identity/v3/domains => generated 190 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:44:42.303975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.357150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68954c-0294-4df7-9ce4-00b39f4d77ac None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.390989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.391626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.391907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.392180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] PATH_INFO: `/v3/users/332151c44437481cb5ef58e0902276ac` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.393894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] RBAC: Authorizing `identity:get_user(user_id=332151c44437481cb5ef58e0902276ac)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.395387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.397381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 9/17] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:42 2022] GET /identity/v3/users/332151c44437481cb5ef58e0902276ac => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:42.402712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.405291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c35911b-7491-4ea0-b60f-8c27c85bcfcd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.417075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.438273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.439149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.439435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.440240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.458819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.460252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:44:42.476075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:42.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:44:42.478305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:44:42.478305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:44:42.478305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:42.478305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 9/18] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:44:42.482932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.503466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c658e20b-4562-4e31-b4be-090e84609bde None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.507667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.538233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.538890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.539170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.539428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.540575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.542045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.543606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 10/19] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:42.549182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:42.587092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b926a-70f6-4199-be15-032f44186d66 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.620785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.621328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.621611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.621886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.622532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:42.622871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.624267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73150b-651a-4c0b-885e-2f00221a2ef7 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.629881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:42.641561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 10/20] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:42.649255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8a524-2e6a-42f5-8b9a-7e477e1fb4b3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:42.686619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:42.687317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:42.687598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:42.687874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] PATH_INFO: `/v3/domains/default/users/332151c44437481cb5ef58e0902276ac/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:42.690329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=332151c44437481cb5ef58e0902276ac, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:42.692193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96632755-51bb-48aa-a6ab-6aae0c24df8d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:42.890008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 11/21] 10.222.0.60 () {58 vars in 1361 bytes} [Fri Oct 21 14:44:42 2022] PUT /identity/v3/domains/default/users/332151c44437481cb5ef58e0902276ac/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:44:45.095619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90ba6-83ea-499d-b335-6731df5515a8 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.096665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90ba6-83ea-499d-b335-6731df5515a8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.096926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe90ba6-83ea-499d-b335-6731df5515a8 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.098533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 11/22] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:45.107849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8c34c-cc5d-403a-8f81-0d6f8cb64f73 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.108726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8c34c-cc5d-403a-8f81-0d6f8cb64f73 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.109289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb8c34c-cc5d-403a-8f81-0d6f8cb64f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.140761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ceb8c34c-cc5d-403a-8f81-0d6f8cb64f73 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:45.142665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ceb8c34c-cc5d-403a-8f81-0d6f8cb64f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:45.229003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 12/23] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:45.619851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da49b-54e5-4f41-924a-f0c6cf4110c7 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.620518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da49b-54e5-4f41-924a-f0c6cf4110c7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.621004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112da49b-54e5-4f41-924a-f0c6cf4110c7 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.622856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 12/24] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:45.629061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.629590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.630085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.652378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:45.653577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:45.692193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14f3e-1620-473b-a7f7-3217830485e1 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.692486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14f3e-1620-473b-a7f7-3217830485e1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.692735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f14f3e-1620-473b-a7f7-3217830485e1 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.694415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 13/25] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:45.699420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c61e6-d44b-4b75-99c1-309f5da1f5ef None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.699702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c61e6-d44b-4b75-99c1-309f5da1f5ef None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.699974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c61e6-d44b-4b75-99c1-309f5da1f5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.730519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ec8c61e6-d44b-4b75-99c1-309f5da1f5ef None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:45.731895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ec8c61e6-d44b-4b75-99c1-309f5da1f5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:45.764031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:45.769331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 13/26] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:45.770224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:45.851441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 14/27] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:45.852710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63333109-6a98-4746-83e4-f7505bf34ddd None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.853203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63333109-6a98-4746-83e4-f7505bf34ddd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.853482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63333109-6a98-4746-83e4-f7505bf34ddd None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.855759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 15/28] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:45.861171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:45.861968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:45.862244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:45.895376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:45.898530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:45.905762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:45.935615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c28b1ff-218d-4f82-9844-8d5b8fc945de None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:45.998338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 16/29] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:46.003709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.004551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.004991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.005435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.020477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:46.022572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:44:46.039232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:46.040325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:44:46.041333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:44:46.041333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:44:46.041333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:44:46.042671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 14/30] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:44:46.055727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.126620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.194278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:46.210322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1d88e-be9a-46e2-88e5-4b1a657451e0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.249020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9291b90-835b-4ac5-bc31-a206f2647db9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.291079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.292184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.293233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.293880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.300321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.301181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.304287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.304806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.305657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:46.306221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:46.308851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:46.316972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:46.326812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:46.327652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:46.328419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:44:46.332064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:46.334168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 15/31] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:44:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:44:46.335997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.338528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 17/32] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:46.344950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.493559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d707f6-55c5-42b6-ac66-0ffd4cec6bf6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.498342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38694c9-0153-45ce-bdb7-2c9af0730d40 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.517277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:46.531437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.532116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.532436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.532729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.533467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:46.535286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:46.547272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.548687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.550565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.551071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.551943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:46.554567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:46.682396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 16/33] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Oct 21 14:44:46 2022] POST /identity/v3/roles => generated 225 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:46.683850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.760355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 18/34] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:44:46 2022] POST /identity/v3/projects => generated 301 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:46.761295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.817959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c056cb-c3e2-4eb0-a926-245a6259b430 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.837518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:46.869135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:46.870082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:46.871033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:46.871033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:46.871334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:46.871624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:46.873207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:46.884781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 17/35] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:44:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:46.891337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:46.916734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed0a67-aea8-4b40-bb23-ea6e1cfe3100 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:46.979739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:47.009994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.010712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.010953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.011230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.011994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.013906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21164ad3-d7f8-4f69-b17a-f45b5a90538f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.090479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd6493-491d-4342-bf1c-c4f9784637ce None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:47.131470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.132199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.132454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.132736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] PATH_INFO: `/v3/users/332151c44437481cb5ef58e0902276ac` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.134791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] RBAC: Authorizing `identity:get_user(user_id=332151c44437481cb5ef58e0902276ac)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.136248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f05ea38-0a09-4d3e-82db-5b15221c19a5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.138245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 18/36] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:46 2022] GET /identity/v3/users/332151c44437481cb5ef58e0902276ac => generated 247 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:47.143618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6c47-386b-4753-b8a5-1eb573a3ce4f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.143618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6c47-386b-4753-b8a5-1eb573a3ce4f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.143618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c6c47-386b-4753-b8a5-1eb573a3ce4f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.143618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 19/37] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:47.144236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6c47-386b-4753-b8a5-1eb573a3ce4f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:47.148677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 19/38] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Oct 21 14:44:46 2022] POST /identity/v3/roles => generated 219 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:47.154071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14071db4-5cca-4aba-842c-61d2fbe146c6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.154423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14071db4-5cca-4aba-842c-61d2fbe146c6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.154701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14071db4-5cca-4aba-842c-61d2fbe146c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.185470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-14071db4-5cca-4aba-842c-61d2fbe146c6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:47.187434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-14071db4-5cca-4aba-842c-61d2fbe146c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:47.293813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 20/39] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:44:47.293813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b00c5-930e-4ee4-aae7-54778168c921 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.293813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b00c5-930e-4ee4-aae7-54778168c921 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.293813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b00c5-930e-4ee4-aae7-54778168c921 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.293813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 21/40] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:47.302588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.302988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.303350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.323825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:47.325072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:47.370992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:47.387752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c6c47-386b-4753-b8a5-1eb573a3ce4f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:47.390854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:47.424445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 22/41] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:44:47.444790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.445451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.445737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.446011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.447180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.448639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.450386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 20/42] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:47.456979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:47.590049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9644c48-d6fc-4233-985c-245e228fde80 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:47.628013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:47.629840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.630677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.631238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.631710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.632709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.domain.id=default, user.id=332151c44437481cb5ef58e0902276ac) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:47.633195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.634980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.650687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 23/43] 10.222.0.60 () {56 vars in 1322 bytes} [Fri Oct 21 14:44:47 2022] GET /identity/v3/role_assignments?user.id=332151c44437481cb5ef58e0902276ac&scope.domain.id=default&role.id=55a557362b9542629b4209da47492ebf => generated 530 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:47.727859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:47.745705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a1d647-994a-4385-b5ba-d7affa7d6863 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:47.772988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:47.801732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.802368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.802666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.802960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.803649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.805207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664c679-45a5-47c7-91f7-1daf6872d95d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.861106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eeb5fa-8183-4a21-a7c1-99c041e6a6cb None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:47.897482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:47.898137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:47.898414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:47.898712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:47.899808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:47.901252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:47.902886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 24/44] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:44:47.908501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:47.917320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 21/45] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Oct 21 14:44:47 2022] POST /identity/v3/roles => generated 223 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:48.020867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90017825-d7b8-4798-8de8-4f8914959444 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:48.023659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:48.058928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:48.059559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:48.059838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:48.060110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:48.060775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:48.062240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48e63c2-bfc6-483c-8ace-46e620a8b4e1 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:48.155623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 25/46] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:44:47 2022] POST /identity/v3/projects => generated 312 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:44:50.297582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c70e8f-54ef-4003-b337-54b6bb69e6cc None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:50.298067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c70e8f-54ef-4003-b337-54b6bb69e6cc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:50.298067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c70e8f-54ef-4003-b337-54b6bb69e6cc None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:50.299992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 22/47] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:50.305167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:50.305557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:50.305735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:50.324206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:50.325388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:50.389326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 26/48] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:50.657981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c70e8f-54ef-4003-b337-54b6bb69e6cc None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:50.688951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:50.758059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c70e8f-54ef-4003-b337-54b6bb69e6cc None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:50.791280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:50.791905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:50.792193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:50.792467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:50.793597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:50.795080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac50681-88ee-4547-a350-8c8becddf756 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:50.796636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 23/49] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:50.803007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:50.885143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:50.899881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79a8f8-562f-4211-95ff-6536d9877746 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:50.931664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:50.932259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:50.932542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:50.932816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:50.933472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:50.934942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def0b3d4-7d2b-4a2f-a85b-c2b6449b7cc2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:51.052812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 27/50] 10.222.0.60 () {60 vars in 1140 bytes} [Fri Oct 21 14:44:50 2022] POST /identity/v3/projects => generated 298 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:53.199323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222869b-7628-4774-987b-852164e4d09d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:53.199827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222869b-7628-4774-987b-852164e4d09d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:53.199909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a222869b-7628-4774-987b-852164e4d09d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:53.201226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 24/51] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:53.206446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:53.206831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:53.207048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:53.225371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:53.226558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:53.293106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 28/52] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:53.563077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222869b-7628-4774-987b-852164e4d09d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:53.657374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:53.661993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a222869b-7628-4774-987b-852164e4d09d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:53.693915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:53.694529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:53.694833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:53.695106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:53.696211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:53.697713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69d1f83-3453-429b-9487-1f2eaea8decf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:53.699276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 25/53] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:53.704752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:53.770086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:53.801787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d930d-2d5e-48e8-8a1f-3b1c0491df3f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:53.940478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:53.941084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:53.941365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:53.941690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:53.942348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:53.944111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6277a8-18a6-4f7f-9e09-7022e71386f5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:54.036603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 29/54] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:44:53 2022] POST /identity/v3/users => generated 275 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:57.360704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f1aa-fec9-4fb8-bc0b-3126d9cf4317 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.362500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f1aa-fec9-4fb8-bc0b-3126d9cf4317 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.363080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6f1aa-fec9-4fb8-bc0b-3126d9cf4317 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.365003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 26/55] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:57.370504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9e415-a15d-4862-b787-89e04fcfad73 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.371394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9e415-a15d-4862-b787-89e04fcfad73 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.371862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9e415-a15d-4862-b787-89e04fcfad73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.395675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f6c9e415-a15d-4862-b787-89e04fcfad73 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:57.397482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f6c9e415-a15d-4862-b787-89e04fcfad73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:57.496581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 30/56] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:44:57.577746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29c96-8639-4a36-8048-e5c3d015d8d2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.578383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29c96-8639-4a36-8048-e5c3d015d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.578908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b29c96-8639-4a36-8048-e5c3d015d8d2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.580966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 27/57] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:57.586946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519b0e-18e6-4a58-8b37-9e7bfb591d0d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.587238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519b0e-18e6-4a58-8b37-9e7bfb591d0d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.587504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519b0e-18e6-4a58-8b37-9e7bfb591d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.616023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-48519b0e-18e6-4a58-8b37-9e7bfb591d0d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:57.617235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-48519b0e-18e6-4a58-8b37-9e7bfb591d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:57.684752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:57.709673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 31/58] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:57.796886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60efabc-0bf9-47ea-98b0-459580b79017 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.797510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60efabc-0bf9-47ea-98b0-459580b79017 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.798022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60efabc-0bf9-47ea-98b0-459580b79017 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.800569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 28/59] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:57.805987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29191-6f68-44f6-8612-5f0a73c2faaa None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:57.806497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29191-6f68-44f6-8612-5f0a73c2faaa None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:57.806990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b29191-6f68-44f6-8612-5f0a73c2faaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:57.826032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c0b29191-6f68-44f6-8612-5f0a73c2faaa None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:57.827529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c0b29191-6f68-44f6-8612-5f0a73c2faaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:57.919633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 32/60] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:57.938667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60efabc-0bf9-47ea-98b0-459580b79017 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.017156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2b6a-c2a1-49b8-a585-b485a9c5cd10 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.017498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2b6a-c2a1-49b8-a585-b485a9c5cd10 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.018258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f2b6a-c2a1-49b8-a585-b485a9c5cd10 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.020017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 33/61] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:44:58.027266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.027545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.027822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.051634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:58.052826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:58.104823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60efabc-0bf9-47ea-98b0-459580b79017 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.149775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 34/62] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:44:58.150411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.151064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.151326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.151743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.152151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.164779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:58.170075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:58.170976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:44:58.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:44:58.174948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 29/63] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:44:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:44:58.184733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.218133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.218834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.280789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc680d12-af3c-47db-99af-99a6113e48f7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.301093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34bcec-3c17-4d2b-bb04-7c0242bc72fc None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.319746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.320396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.320678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.320952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.334596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:58.337207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.337822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.338107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.338388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.339084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:58.339395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-5b5c8a55-4ccf-46fc-a432-f5e001ab5746 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:58.340678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:58.341447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:44:58.342195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:58.342784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.342858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 35/64] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:44:58.343829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90e58-82f4-4a0b-b7cc-b19b1773d3b2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.344112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90e58-82f4-4a0b-b7cc-b19b1773d3b2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.344388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a90e58-82f4-4a0b-b7cc-b19b1773d3b2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.346291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 36/65] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:44:58.347142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a90e58-82f4-4a0b-b7cc-b19b1773d3b2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.360000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 30/66] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.360000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.454690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.459797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a90e58-82f4-4a0b-b7cc-b19b1773d3b2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.462132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.467258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a523bd5-0cb0-49bb-b056-3ce413cac004 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.494078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.494742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.495023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.495299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.502572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.503210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.503495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.503773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.504444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:58.504754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.505387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:58.506123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-4054e522-8790-4097-8457-23c4348bb7b1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:58.510488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:58.511359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 37/67] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:44:58.517368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.518907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.518907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.519020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 31/68] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.519020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.534357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:44:58.535616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:44:58.601549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 38/69] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:44:58.602375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.615664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf15f11-31e2-49c5-b8bb-e2eb5e6b2c22 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.653450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.654313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.654597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.654906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] PATH_INFO: `/v3/users/332151c44437481cb5ef58e0902276ac` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.656062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] RBAC: Authorizing `identity:get_user(user_id=332151c44437481cb5ef58e0902276ac)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.657498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.660303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 32/70] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/users/332151c44437481cb5ef58e0902276ac => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.661105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.681779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.714536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.722619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fa2f5e-cd7e-495a-9c23-5bede64f0153 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.757359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.758005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.758288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.758561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.759715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.761174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.762910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 39/71] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.763518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.770172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753972d2-9c43-4c16-b6f3-0a26f7dd953b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.806744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.807414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.808050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.808050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.808788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:58.809208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.810673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.819428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 33/72] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.820273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.871694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb2c6-55d7-466e-a5d4-906e44498890 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.906180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.906660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.907751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.908035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.908346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.923578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.924097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fccd28c-1935-4895-ac72-9a66c6018938 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:58.925268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.926946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 40/73] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.927702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:58.951550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:58.959489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:58.960126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:58.960409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:58.960681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:58.961816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:58.963306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:58.967266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 34/74] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:58.967266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.031259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555946d-3099-4dd5-8b7e-3bf1088a6ea9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.065695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.066320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.066603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.066914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.067335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a263f-56d6-45fb-8135-e92f8171dbef None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.067569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.069021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.100694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.101322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.101605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.101881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.103057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.104492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.106060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 35/75] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:58 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:59.106846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.124204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.151292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 41/76] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:44:58 2022] POST /identity/v3/users => generated 289 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:44:59.152171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.216007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4af43-6be5-4f4f-b14f-af9422686d9e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.254079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.254687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.254979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.255257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.256481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.266456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:44:59.272554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:44:59.273477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:44:59.274740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 36/77] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:44:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:44:59.275654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.288922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d09817-6a27-4e71-8646-37bd531c49cb None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.330386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.331827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.331827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.331827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.333342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.334829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.336497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 42/78] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:44:59 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:59.342085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.373279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.400126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813335e5-1e44-4db1-bf0b-decbf11f90dd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.442988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.443908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.444193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.444479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/332151c44437481cb5ef58e0902276ac/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.446844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, user_id=332151c44437481cb5ef58e0902276ac, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.449720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.459293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f361c3c-fd10-4cae-a333-2a5562095e46 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.496327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.497785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.498124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.498408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.499629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.501122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.503585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 43/79] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:44:59 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:59.504252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.519921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.522494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 37/80] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:44:59 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/332151c44437481cb5ef58e0902276ac/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:44:59.527602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.700972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49505ab0-843b-42e0-b80a-41e5df628bcc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.725229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd6c1f7-2ef0-41ad-882f-bb84be761f48 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.752716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.753389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.753676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.753953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.754728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:44:59.754987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.756376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.768114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.769255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.769742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.770272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/ee30fc223ad243bda597e1528974c751/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.773375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 44/81] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:44:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:44:59.773974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.775261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, user_id=ee30fc223ad243bda597e1528974c751, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.777433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:44:59.782962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.867651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:44:59.890540 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 38/82] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:44:59 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/ee30fc223ad243bda597e1528974c751/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:44:59.891684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:44:59.894382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2011c3f-b8c6-4a0f-b708-f4ea4f4c8c20 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:44:59.944374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:44:59.945310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:44:59.946678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:44:59.946678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/ee30fc223ad243bda597e1528974c751/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:44:59.954659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, user_id=ee30fc223ad243bda597e1528974c751, role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:44:59.954855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:00.047469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 45/83] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:44:59 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/ee30fc223ad243bda597e1528974c751/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:00.091717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee7f49-fb73-496a-a977-e450be04ec1a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:00.131960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:00.133010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:00.133487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:00.133977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:00.135424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:00.137143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:00.139447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 39/84] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:44:59 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:00.145425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:00.272508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2c1696-7d3c-4531-8196-060a35ca53c2 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:00.313225 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:00.313875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:00.314163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:00.314447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] PATH_INFO: `/v3/projects/8e90b897a0f54599aea8ed7ce6181648` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:00.316382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] RBAC: Authorizing `identity:get_project(project_id=8e90b897a0f54599aea8ed7ce6181648)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:00.317863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf9a9d-f09b-41db-816c-4cb04c4401a9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:00.319589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 46/85] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:00 2022] GET /identity/v3/projects/8e90b897a0f54599aea8ed7ce6181648 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:00.325604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:00.425998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:00.476079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ba85-2c59-477a-ab09-08143e16c2cd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:00.521039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:00.521957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:00.522237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:00.522508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] PATH_INFO: `/v3/projects/8e90b897a0f54599aea8ed7ce6181648/users/ee30fc223ad243bda597e1528974c751/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:00.524487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] RBAC: Authorizing `identity:create_grant(project_id=8e90b897a0f54599aea8ed7ce6181648, user_id=ee30fc223ad243bda597e1528974c751, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:00.525924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e63be5-feae-44bf-a090-3d3d93b4cd7e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:00.617725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 40/86] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:00 2022] PUT /identity/v3/projects/8e90b897a0f54599aea8ed7ce6181648/users/ee30fc223ad243bda597e1528974c751/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:02.027464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aae5c5-17ae-4c1a-863a-4a229fb48a25 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.027969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aae5c5-17ae-4c1a-863a-4a229fb48a25 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.028040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aae5c5-17ae-4c1a-863a-4a229fb48a25 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.029365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 47/87] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:02.039982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.040392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.040574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.066862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:02.068075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:02.154122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553bef-9e0d-4098-a6c5-b900764c27f8 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.154436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553bef-9e0d-4098-a6c5-b900764c27f8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.154737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e553bef-9e0d-4098-a6c5-b900764c27f8 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.156076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 48/88] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:02.158518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 41/89] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:02.161790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.162080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.162364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.176391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:02.195615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:02.196839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:02.282180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 49/90] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:02.694039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:02.773020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:02.814094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:02.844506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c3d8e-0dca-4a37-9636-b374660fe03b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:02.910852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:02.911530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.911814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.912090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.934686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-63fdb358-40ef-4434-8cd3-ea88f8bf19f7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:02.940849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:02.941746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:02.942507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 42/91] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:02.943164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7b29-1e39-48de-beed-794ec8b25d7e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.943955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7b29-1e39-48de-beed-794ec8b25d7e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.943955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da7b29-1e39-48de-beed-794ec8b25d7e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.945065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 43/92] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:02.958362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd3282-c556-4f1f-9a95-e5e24cee057e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:02.958654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd3282-c556-4f1f-9a95-e5e24cee057e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:02.958942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd3282-c556-4f1f-9a95-e5e24cee057e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:02.997257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-76dd3282-c556-4f1f-9a95-e5e24cee057e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:02.998470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-76dd3282-c556-4f1f-9a95-e5e24cee057e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:03.004481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba238-5bb4-416c-b87a-26b4f7b69a6d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.044785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:03.049959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.050673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.050973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.051250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.052484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:03.053992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:03.055906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 50/93] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:03.056793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.092799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 44/94] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:03.095378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630de24-f5a3-4074-b5c1-5d3297020a1d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.095672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630de24-f5a3-4074-b5c1-5d3297020a1d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.096512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5630de24-f5a3-4074-b5c1-5d3297020a1d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.097949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 45/95] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:03.098538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630de24-f5a3-4074-b5c1-5d3297020a1d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.168731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:03.182023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcbcdb-4318-436d-80e7-cd366db0a980 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.218397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.219016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.219304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.219689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.220367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:03.220674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:03.222947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8938da-6cad-493f-aa62-043416674da9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:03.231905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 51/96] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:03.233075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.233400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.233959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.258890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:03.260144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:03.275690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5630de24-f5a3-4074-b5c1-5d3297020a1d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.326509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.327256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.327629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.327909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.328573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:03.330029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:03.368707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:03.380322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 52/97] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:03.381212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.442758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 46/98] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:45:03 2022] POST /identity/v3/projects => generated 302 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:03.443619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.575659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78506188-4a29-48c8-aa95-f1ff994f079f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.643426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.644237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.644519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.644792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] PATH_INFO: `/v3/users/2c686b5c38a24ca2b1909bab3eca6443` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.648188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:03.661004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c37db5-2932-4f02-9ad2-fc66cb10a8b6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.677359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] RBAC: Authorizing `identity:get_user(user_id=2c686b5c38a24ca2b1909bab3eca6443)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:03.677359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:03.678624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 53/99] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/users/2c686b5c38a24ca2b1909bab3eca6443 => generated 289 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:03.687168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.730509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751970c-e396-4641-b32f-43d655675135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.731192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751970c-e396-4641-b32f-43d655675135 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.731479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751970c-e396-4641-b32f-43d655675135 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.731754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3751970c-e396-4641-b32f-43d655675135 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.758207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3751970c-e396-4641-b32f-43d655675135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3751970c-e396-4641-b32f-43d655675135 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:03.763230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:03.764241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:03.765207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:03.769926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 47/100] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:03.770895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751970c-e396-4641-b32f-43d655675135 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.784523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:03.845873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c32a34-29ef-43d9-a248-cfe037517ddf None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.887859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3751970c-e396-4641-b32f-43d655675135 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:03.889674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.890371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.891691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.891971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.893272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:03.894820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:03.896560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 54/101] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:03.898738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.929096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:03.929739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.930058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.930354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.942669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c885c8e5-1ff3-4f1b-8471-12a4341ba982 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:03.947768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:03.949527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 48/102] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:03.951442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5fc5-154e-46c6-9a97-0458b0aefc8a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:03.953067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5fc5-154e-46c6-9a97-0458b0aefc8a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:03.953067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b5fc5-154e-46c6-9a97-0458b0aefc8a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:03.953067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 49/103] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:03.953444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5fc5-154e-46c6-9a97-0458b0aefc8a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:03.973485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.026932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de3dfc-0d06-45d0-a5ab-74cc86693fb3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.045092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.062837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.063454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.063738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.064007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.065272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:04.065582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.067006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.075253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 55/104] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:04.075376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b5fc5-154e-46c6-9a97-0458b0aefc8a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.080600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:04.112707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.113434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.113720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.114002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/2c686b5c38a24ca2b1909bab3eca6443/roles/81ae57558f38464ebce24e926397f7e9` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.115978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, user_id=2c686b5c38a24ca2b1909bab3eca6443, role_id=81ae57558f38464ebce24e926397f7e9)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.117699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77ad1e1-f5f2-4880-820b-26a665adbcd6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.183496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dafc9-bade-4911-add4-cdb356da6b26 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.206666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 50/105] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:03 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/users/2c686b5c38a24ca2b1909bab3eca6443/roles/81ae57558f38464ebce24e926397f7e9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:04.206666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.206666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.206666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.220256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:04.222123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:04.232228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.232933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.233216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.233597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.234289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:04.234596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.236260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3e9c9-a004-4478-ab7a-98d19d62e41e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 56/106] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc110d1-2d18-4eb5-929e-b93133f562a7 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc110d1-2d18-4eb5-929e-b93133f562a7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc110d1-2d18-4eb5-929e-b93133f562a7 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 57/107] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:04.252915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc110d1-2d18-4eb5-929e-b93133f562a7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:04.316291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.323080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 51/108] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:04.324398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:04.333899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.391473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc110d1-2d18-4eb5-929e-b93133f562a7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.431263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.432266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.432770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.433307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] PATH_INFO: `/v3/users/332151c44437481cb5ef58e0902276ac` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.434772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] RBAC: Authorizing `identity:get_user(user_id=332151c44437481cb5ef58e0902276ac)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.436533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec01468-f256-4fc0-ac19-1730ac518f96 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.438409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 58/109] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:04 2022] GET /identity/v3/users/332151c44437481cb5ef58e0902276ac => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:04.440703 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.441291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.441790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.448648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbffb3b-ab5d-4ee2-bd91-9092c825a587 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.463081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:04.464663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:04.490871 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.491519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.491805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.492393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.493767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.495784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.497644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 52/110] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:04 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:04.502790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:04.545819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 59/111] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:04.547031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:04.663109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.675369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:04.723818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651693-4b4b-4d2e-82a7-df1f4b7524c5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.780549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd2b2-8cf9-4e4e-bb91-f9d4719c50fc None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:04.786546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.787189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.787469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.790823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.863396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:04.864029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:04.864306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:04.864579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:04.891310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.892827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:04.987309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:04.989453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.006743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.040154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 53/112] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:05.041325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.047338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.124854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 60/113] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:05.127811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.175097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65acbe6a-e000-4ab1-9120-408c91b723ed None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.218671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.219796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.221039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.221528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.234017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:05.240330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:05.241178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:05.241929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:05.244068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 54/114] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:05.245168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.263951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ea775-af03-4de8-8caa-9dbb5e81ee0d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.289498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.308978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.309616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.309894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.313260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.320209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.330661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:05.333948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:05.334836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:05.335597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 61/115] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:05.342657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.372823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebbefd8-9cea-4590-a47b-56ba91046cf4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.409906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.410816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.411241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.411656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.413083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.414709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.416506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 55/116] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.418528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.448056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1349b053-aa4f-4bc5-bde5-04cd8725728f None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.485835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.487025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.487462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.487741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.488891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.491088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.497354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 62/117] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.497354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.536386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43ebd-949f-43aa-b093-195d295a547e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.538235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.571539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.575073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.576940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.577396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.577823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.578719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:05.579190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.580772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.589952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 56/118] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.595852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.616357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7924c11-b6a7-4d11-a364-69d97382f79c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.660965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.662249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.662950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.663373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.664292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] RBAC: Adding query filter params (name=member) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:05.664819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.666364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.677139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 63/119] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.682962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.716024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ba6c9-6e59-44d6-abe1-5d6e3dcae05d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.762665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.763767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.764049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.764588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.765329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79, user.id=332151c44437481cb5ef58e0902276ac) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:05.765641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.769458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.778893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 57/120] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/role_assignments?user.id=332151c44437481cb5ef58e0902276ac&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.780097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:05.804600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c437c6-2018-45e1-9d59-903b1438025c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:05.827411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.857020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:05.869475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:05.870124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:05.870415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:05.870738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:05.871588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] RBAC: Adding query filter params (role.id=e6c31a1ad1c74840b2b04e21d9ff67bf, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79, user.id=ee30fc223ad243bda597e1528974c751) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:05.871907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:05.873471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:05.888819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 64/121] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/role_assignments?user.id=ee30fc223ad243bda597e1528974c751&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=e6c31a1ad1c74840b2b04e21d9ff67bf => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:05.890329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.018723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b3cae-ba5b-4c0b-a336-ee1f0e487a75 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.064616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.065358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.065638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.065911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.067265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.069472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cae110-fbf1-455e-a7a1-4606529903b8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.071916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 58/122] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:06.072880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068270ce-1b61-4a10-964b-a98a87a508a2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.073160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068270ce-1b61-4a10-964b-a98a87a508a2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.073431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068270ce-1b61-4a10-964b-a98a87a508a2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.075290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c81694e-0d4a-40de-9f03-5fc9e6b22eba None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.082089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 59/123] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:06.082089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068270ce-1b61-4a10-964b-a98a87a508a2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.119412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.120485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.120964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.121414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] PATH_INFO: `/v3/users/ee30fc223ad243bda597e1528974c751` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.122914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] RBAC: Authorizing `identity:get_user(user_id=ee30fc223ad243bda597e1528974c751)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.124612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740d046-52e9-4579-8f2b-58b0bb2bf71f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.126508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 65/124] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:05 2022] GET /identity/v3/users/ee30fc223ad243bda597e1528974c751 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:06.129029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.129532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.130004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.152981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:06.154537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:06.192538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.200798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068270ce-1b61-4a10-964b-a98a87a508a2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.229422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.245998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 66/125] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:06.247792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.248586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.249049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.249443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.256163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.344080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.346768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.371791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216ad3c-b5e3-485e-94b7-397c72049085 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.411857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.412484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.412759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.413096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.462015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.470616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 60/126] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:06.476788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.481883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.494297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.496179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.584824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692172cb-0c81-4f1d-8987-89b98867c73c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.609799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 67/127] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:06.610419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.625890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.626638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.627462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.627740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.628879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.630406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.636962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 61/128] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:06.637745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.703963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.709317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.725531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599a1fc9-caf6-4ce0-92a9-59adb5496abc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.742476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662c2c2-7961-47c0-9d2d-bf8742cfc638 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.751280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.752444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.752795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.754729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.754729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.755709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.757295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 68/129] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:06.758065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.777946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.778583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.778890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.779159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] PATH_INFO: `/v3/projects/8e90b897a0f54599aea8ed7ce6181648` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.780272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] RBAC: Authorizing `identity:get_project(project_id=8e90b897a0f54599aea8ed7ce6181648)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.781702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f008587-3efa-4a66-8ada-f2322c286476 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.783857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 62/130] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/projects/8e90b897a0f54599aea8ed7ce6181648 => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:06.784965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d152-f456-46ec-8d32-3bc8cf949e9a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.785244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d152-f456-46ec-8d32-3bc8cf949e9a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.785516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7d152-f456-46ec-8d32-3bc8cf949e9a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.786720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 63/131] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:06.787402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7d152-f456-46ec-8d32-3bc8cf949e9a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.867879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a246b9e5-fb97-483d-a069-7d8ed7a2eb86 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.889372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7d152-f456-46ec-8d32-3bc8cf949e9a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:06.903414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.904064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.904333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.904613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.905350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] RBAC: Adding query filter params (role.id=8a04c4ea718349e293dc17454cef7b86, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79, user.id=ee30fc223ad243bda597e1528974c751) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:06.905661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.907138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.912767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 69/132] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/role_assignments?user.id=ee30fc223ad243bda597e1528974c751&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:06.913560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:06.919718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:06.927207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:06.932888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:06.932888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:06.932888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:06.932888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:06.932888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db2141-d371-4915-b887-8f954e39ace8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:06.942256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:07.024894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eea500-82a1-4325-ae62-354eb82401bc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:07.052904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 64/133] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:06 2022] POST /identity/v3/users => generated 283 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:07.054275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:07.054567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:07.054864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:07.070664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:07.070888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:07.078661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:07.078661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:07.078661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] RBAC: Adding query filter params (role.id=8a04c4ea718349e293dc17454cef7b86, scope.project.id=8e90b897a0f54599aea8ed7ce6181648, user.id=ee30fc223ad243bda597e1528974c751) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:07.078661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:07.078661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:07.079659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:07.079659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:07.082477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 70/134] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:06 2022] GET /identity/v3/role_assignments?user.id=ee30fc223ad243bda597e1528974c751&scope.project.id=8e90b897a0f54599aea8ed7ce6181648&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:07.171690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 65/135] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:07.451031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:07.549356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:07.562742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443588e-804d-4dc4-8442-7d1095cf9e3e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:07.599448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:07.600158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:07.600449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:07.600730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:07.616145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:07.621544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:07.622372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:07.623182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 71/136] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:07.630717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:07.736890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea5a968-ad75-47bf-a12d-8a0a1d58ccc1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:07.756589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:07.774980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:07.775622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:07.775907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:07.776185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:07.776866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:07.777176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:07.778581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:07.787777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 66/137] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:07.793608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:07.905229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fe475-9f3b-47b2-8350-e39a8f8f76eb None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:07.940985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:07.941626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:07.941909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:07.942184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] PATH_INFO: `/v3/users/2c686b5c38a24ca2b1909bab3eca6443` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:07.943351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] RBAC: Authorizing `identity:get_user(user_id=2c686b5c38a24ca2b1909bab3eca6443)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:07.944808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:07.946746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 72/138] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:07 2022] GET /identity/v3/users/2c686b5c38a24ca2b1909bab3eca6443 => generated 289 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:07.957303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:08.070816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65aa2f5-056c-4639-8b60-9473d12dca09 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:08.106706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:08.107328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:08.107612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:08.107888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:08.163579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:08.184650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:08.186144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:08.293382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 67/139] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:08.299762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:08.323094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:08.411205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86e6c1-d600-4e81-8b15-c3481362db12 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:08.445536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:08.446178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:08.446462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:08.446768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:08.447909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:08.449368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c754d1c-f90d-4b41-87d7-552220d95f8a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:08.451027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 73/140] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:08 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:08.457787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:08.546534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:08.565987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02755b53-df94-46a6-aab8-76c4bba80f94 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:08.600907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:08.601739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:08.602217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:08.602657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:08.603776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] RBAC: Adding query filter params (role.id=81ae57558f38464ebce24e926397f7e9, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79, user.id=2c686b5c38a24ca2b1909bab3eca6443) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:08.604244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:08.605808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba363e4c-9949-4500-a379-cb1e6648eaf6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:08.611951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 68/141] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:08 2022] GET /identity/v3/role_assignments?user.id=2c686b5c38a24ca2b1909bab3eca6443&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=81ae57558f38464ebce24e926397f7e9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:09.465538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed813bf-17a4-4362-857a-fbd71fb8c63d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.466077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed813bf-17a4-4362-857a-fbd71fb8c63d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.466077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed813bf-17a4-4362-857a-fbd71fb8c63d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.467516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 74/142] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:09.473103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d380a8-3ef1-496a-a061-378e66f4432a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.473520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d380a8-3ef1-496a-a061-378e66f4432a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.473708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d380a8-3ef1-496a-a061-378e66f4432a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.492923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-22d380a8-3ef1-496a-a061-378e66f4432a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:09.494126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-22d380a8-3ef1-496a-a061-378e66f4432a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:09.518543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975645-7c24-4f88-9751-7e993593f581 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.518867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975645-7c24-4f88-9751-7e993593f581 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.519152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975645-7c24-4f88-9751-7e993593f581 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.520543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 75/143] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:09.526704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94167-20f3-4ee2-b9f3-1a8fed2e039b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.526988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94167-20f3-4ee2-b9f3-1a8fed2e039b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.527266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b94167-20f3-4ee2-b9f3-1a8fed2e039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.546120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-26b94167-20f3-4ee2-b9f3-1a8fed2e039b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:09.547590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-26b94167-20f3-4ee2-b9f3-1a8fed2e039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:09.570025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 69/144] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:09.624001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 76/145] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:09.650654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433472-391c-49f4-9b65-3e2e41154a5f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.650953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433472-391c-49f4-9b65-3e2e41154a5f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.651229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27433472-391c-49f4-9b65-3e2e41154a5f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.652584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 70/146] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:09.657727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.658016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.658474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:09.676381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:09.677577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:09.687337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:09.744435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 77/147] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:09.836584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433472-391c-49f4-9b65-3e2e41154a5f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:09.905401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:09.917074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:09.945873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27433472-391c-49f4-9b65-3e2e41154a5f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:09.991889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:09.992614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:09.992809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:09.993071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.004852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:10.009728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:10.011054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:10.012074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 71/148] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:10.013259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.017097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39ec12e-3227-4209-bc71-70f7fe25e19a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.053009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.053636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.053909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.054175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.064305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:10.069391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:10.070232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:10.071118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 78/149] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:10.071752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.076273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.116008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea40f39-7bb7-4c3f-b248-8214af3acada None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.149989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.150607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.150906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.151175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.152275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.153748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.155338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 72/150] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:10.160509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.160509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.177377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5b9391-e41e-47de-a420-349bd04d3b48 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.210153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.210784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.211068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.211337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.211995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:10.212303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.213664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.222333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 79/151] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:10.222967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.259124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559b5f6-0c93-45be-a2e4-20dd1ea8218c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.291502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.292016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8a85-466e-4672-8580-55847f761366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.292824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8a85-466e-4672-8580-55847f761366 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.293256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8a85-466e-4672-8580-55847f761366 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.293529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c8a85-466e-4672-8580-55847f761366 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.294349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c8a85-466e-4672-8580-55847f761366 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:10.294675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c8a85-466e-4672-8580-55847f761366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.296063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c8a85-466e-4672-8580-55847f761366 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.304530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 73/152] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:10.306033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8a85-466e-4672-8580-55847f761366 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.325748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd09fc-0a63-4370-813e-a8a5ecb00db9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.359675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.360288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.360567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.360844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.362064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.363544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.367497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.404163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c8a85-466e-4672-8580-55847f761366 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.438070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.438740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.439024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.439300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] PATH_INFO: `/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.452728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] RBAC: Authorizing `identity:get_user(user_id=c1ca9cf7f06f4dacb2c28fb7701984a8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.454184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.456077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 74/153] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8 => generated 283 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:10.456862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.474433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 80/154] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:10 2022] POST /identity/v3/users => generated 297 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:10.474433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.566249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3918e9-4d3e-45e6-acb3-1d118da5d66e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.582662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9dac5-1b7e-491b-a396-a146bcfa312b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.594219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.606955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.607610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.607891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.608161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] PATH_INFO: `/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.609296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.610127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] RBAC: Authorizing `identity:get_user(user_id=c1ca9cf7f06f4dacb2c28fb7701984a8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.611726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.613720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 75/155] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8 => generated 283 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:10.619441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.621730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.622386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.622688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.622964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.624196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.625684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.627494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 81/156] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:10.634105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.742855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7426f9-2e88-4422-a877-091895775167 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.765744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2be39-742f-40c7-92b6-770d10b09e69 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.791834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.793008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.793294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.793827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.795326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.796796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.798662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 76/157] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:10 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:10.805019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:10.811169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.811456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.811729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.811997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/c1ca9cf7f06f4dacb2c28fb7701984a8/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.814114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, user_id=c1ca9cf7f06f4dacb2c28fb7701984a8, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.815651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039717fb-c673-4b45-a031-a59b41d28e02 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:10.870989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:10.890671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 82/158] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:10 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/c1ca9cf7f06f4dacb2c28fb7701984a8/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:10.924865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b1f49-d8e6-431d-b48c-0e88aa758767 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:10.971434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:10.972103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:10.972374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:10.972642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/c1ca9cf7f06f4dacb2c28fb7701984a8/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:10.975318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, user_id=c1ca9cf7f06f4dacb2c28fb7701984a8, role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:10.976865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e65c7-cf3d-4979-8f48-109c07fba45a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:11.039401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 77/159] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:10 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/c1ca9cf7f06f4dacb2c28fb7701984a8/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:12.768100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2fb83-a565-49db-ad6d-369e4b6903ef None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:12.768720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2fb83-a565-49db-ad6d-369e4b6903ef None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:12.768720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb2fb83-a565-49db-ad6d-369e4b6903ef None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:12.770013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 83/160] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:12.775575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734517-7a98-4566-a2c7-c02aaaac669d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:12.775995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734517-7a98-4566-a2c7-c02aaaac669d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:12.776167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8734517-7a98-4566-a2c7-c02aaaac669d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:12.795536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a8734517-7a98-4566-a2c7-c02aaaac669d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:12.796773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a8734517-7a98-4566-a2c7-c02aaaac669d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:12.873658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 78/161] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:12.946354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e56e-76a8-4c2b-bc0f-f983c19603c2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:12.946731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e56e-76a8-4c2b-bc0f-f983c19603c2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:12.947015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041e56e-76a8-4c2b-bc0f-f983c19603c2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:12.948493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 84/162] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:12.953895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:12.954209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:12.954437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:12.975314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:12.976519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:13.029270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.057408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 79/163] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:13.152435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.152652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96dba-e1ad-453e-b630-d64b9ece667d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.152652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96dba-e1ad-453e-b630-d64b9ece667d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.152652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b96dba-e1ad-453e-b630-d64b9ece667d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.154695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 85/164] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:13.161112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76afe7-fb30-4d12-93a7-7747a0015e88 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.161750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76afe7-fb30-4d12-93a7-7747a0015e88 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.162233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76afe7-fb30-4d12-93a7-7747a0015e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.181868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0b76afe7-fb30-4d12-93a7-7747a0015e88 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:13.183534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0b76afe7-fb30-4d12-93a7-7747a0015e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:13.235377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.255777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 86/165] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:13.262804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c58ba-4c20-4e40-92d6-b60617ee4ca8 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.262804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c58ba-4c20-4e40-92d6-b60617ee4ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.262804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8c58ba-4c20-4e40-92d6-b60617ee4ca8 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.262804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 87/166] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:13.268838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.269121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.269389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.279660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8aa7e6-7bb7-4d56-9dc5-1e0a85d30578 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.291257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:13.292753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:13.320014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.320656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.320930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.321200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.332372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:13.337134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:13.338133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:13.338983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 80/167] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:13.347638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.369344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 88/168] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:13.370256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.389077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.464651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b578c-d0b2-4638-a947-0e85b8f47a1a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.498805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d546e-6f84-4f7b-ae5f-936899d2d96e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.511214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.512297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.512580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.512919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.513597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:13.513932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:13.514482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.516699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:13.530848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 81/169] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:13.534618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.550226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.551060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.551345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.551617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.552828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:13.554254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:13.556028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 89/170] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:13.559894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.642126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.645950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523581c-f827-4da5-8ae1-17dd17433b97 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.667349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba564f-36a3-4671-b9cf-80c6821cdd90 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.681097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.681720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.681995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.682264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] PATH_INFO: `/v3/users/99334971887945669c6f337a2abd686a` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.696773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] RBAC: Authorizing `identity:get_user(user_id=99334971887945669c6f337a2abd686a)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:13.698214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:13.700041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 82/171] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/users/99334971887945669c6f337a2abd686a => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:13.700692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.702815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.703448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.703726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.703997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.714462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:13.719322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:13.720221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:13.721063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 90/172] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:13.723513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.764285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.799569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfef2df-7c9f-4a07-ae6a-babeaa9388dd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.823959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56d97b-7172-469c-a2f3-8abea1659509 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:13.833408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.834002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.834281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.834552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.835243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:13.836687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:13.858333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:13.859010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:13.859291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:13.859565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:13.870117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:13.875140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:13.876033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:13.876887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 91/173] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:13.878109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.950696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 83/174] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:13 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:13.950696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:13.973634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.983665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:13.988924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ae219-2528-4c5a-9786-0a8ac51ee4fe None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.027445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.028126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.028410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.028688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.029891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.031409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.033806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 92/175] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:14.035040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.099095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365a27-c4c3-46bd-8323-a4529e4e8ed0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.138308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.138965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.139246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.139528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.140236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:14.140544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.142998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.152131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 84/176] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:14.152784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.166930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d4fe7-24da-4db7-b86b-acf21969aabe None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.213938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.214952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.215290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.215535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.216507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:14.216507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.217875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.230553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 93/177] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:14.231722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.262021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:14.274623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:14.291611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3176c-6f1a-415a-944a-770119ff5223 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.343512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.343512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.344256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.344530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/99334971887945669c6f337a2abd686a/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.347889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, user_id=99334971887945669c6f337a2abd686a, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.349447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.394691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31215194-cf00-4d51-bd50-3a42fd9e11a9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.438106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.438845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.439121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.439639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] PATH_INFO: `/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.440971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] RBAC: Authorizing `identity:get_user(user_id=c1ca9cf7f06f4dacb2c28fb7701984a8)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.443011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.446067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 94/178] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:14 2022] GET /identity/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:14.450924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.452384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 85/179] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:14 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/99334971887945669c6f337a2abd686a/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:14.453584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.680269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8847-730c-40a0-bf15-ad237e68f7cf None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.688524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:14.695226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94342591-904b-41ee-8c36-89f6ebc6ff72 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.732954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.733539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.733821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.734094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.737938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.738894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.739391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.739502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:14.740110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] PATH_INFO: `/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:14.741528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] RBAC: Authorizing `identity:get_user(user_id=c1ca9cf7f06f4dacb2c28fb7701984a8)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.743230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.745090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 95/180] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:14 2022] GET /identity/v3/users/c1ca9cf7f06f4dacb2c28fb7701984a8 => generated 283 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:14.751143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:14.832118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:14.833606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:14.883556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4974c-a602-4faf-b643-06265d1e7f31 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:14.960707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a3320-780e-400e-933a-473674114022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:14.961265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a3320-780e-400e-933a-473674114022 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:14.961546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a3320-780e-400e-933a-473674114022 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:14.961818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a3320-780e-400e-933a-473674114022 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:15.030553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:15.038952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 86/181] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:15.062189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:15.090539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a3320-780e-400e-933a-473674114022 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:15.092049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538a3320-780e-400e-933a-473674114022 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:15.125524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:15.245715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 96/182] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 21 14:45:15.250529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316722d3-2e05-4136-9db8-fb628d2738df None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:15.251153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a3320-780e-400e-933a-473674114022 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:15.301382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:15.302129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:15.302411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:15.302804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:15.303990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:15.305558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:15.307250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 87/183] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:15 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:15.313063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:15.315867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:15.391406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a3320-780e-400e-933a-473674114022 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:15.452320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3478d4c8-3f23-48d5-9a6a-b23c60e679d5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:15.455157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:15.457970 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:15.458612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:15.458907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:15.459183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:15.463357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:15.464819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:15.466684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 97/184] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:15 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:15.480226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:15.508801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:15.509342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:15.509620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:15.509893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:15.510648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871, user.id=c1ca9cf7f06f4dacb2c28fb7701984a8) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:15.510966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:15.512528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6efc74d-e9a4-43e0-b8ce-87e7b14db4eb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:15.522362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 88/185] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:15 2022] GET /identity/v3/role_assignments?user.id=c1ca9cf7f06f4dacb2c28fb7701984a8&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:15.641408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2430322-ad19-42a9-83a6-0b5a1106c587 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:15.684323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:15.697012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:15.698491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:15.698491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:15.698491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:15.698937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] RBAC: Adding query filter params (role.id=e6c31a1ad1c74840b2b04e21d9ff67bf, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871, user.id=c1ca9cf7f06f4dacb2c28fb7701984a8) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:15.699251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:15.700712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0057aae-a93e-463b-9d8e-54dfb7a7400c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:15.711389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 98/186] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:15 2022] GET /identity/v3/role_assignments?user.id=c1ca9cf7f06f4dacb2c28fb7701984a8&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=e6c31a1ad1c74840b2b04e21d9ff67bf => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:16.600118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1d90-83bc-4253-a0dd-1808b6ae6b60 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:16.600638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1d90-83bc-4253-a0dd-1808b6ae6b60 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:16.600757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d1d90-83bc-4253-a0dd-1808b6ae6b60 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:16.602141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 89/187] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:16.607552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:16.607973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:16.608160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:16.627388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:16.628631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:16.633957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9868-95e7-4b6b-ba6e-1b92b7d40449 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:16.634241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9868-95e7-4b6b-ba6e-1b92b7d40449 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:16.634517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be9868-95e7-4b6b-ba6e-1b92b7d40449 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:16.636022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 90/188] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:16.640659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:16.640944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:16.641212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:16.659379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:16.660560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:16.698186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 99/189] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:16.730431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 91/190] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:16.974922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.003820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.082920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81857110-145e-4b91-b62a-73e909cf05a0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.101127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba79d817-18ae-4a0c-8214-af21f0423f51 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.108320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.116742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.117376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.117658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.117926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.128787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-71bb0c67-209a-4d44-8f06-5c393c3696fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:17.133711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:17.134603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:17.135436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 100/191] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:17.136645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc80-c5bf-41f6-9bb7-dbf6adcd3fe6 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.136718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc80-c5bf-41f6-9bb7-dbf6adcd3fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.136978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338cc80-c5bf-41f6-9bb7-dbf6adcd3fe6 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.138218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 101/192] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:17.143100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.143877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cc80-c5bf-41f6-9bb7-dbf6adcd3fe6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.144001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.144160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.144433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.145566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.147285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e320e-efcb-46a2-8450-16575a49d5dc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.148844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 92/193] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:17.151088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.151376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.151654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.169881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:17.171126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:17.239068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 93/194] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:17.239921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.253363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338cc80-c5bf-41f6-9bb7-dbf6adcd3fe6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.289562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.290170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.290446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.290764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.291400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:17.291836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.293080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.301479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 102/195] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:17.306855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.341331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.347971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fafa3ad-a96e-44d6-a5b9-feddfee42206 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.361433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.383189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.383833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.384178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.384446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.385104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.386550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.415745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91086f4-4b32-47fa-9e77-9dd00f40160b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.448475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.449104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.449383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.449655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] PATH_INFO: `/v3/users/9f1f76c9bdb64c12856c29da8756eaa7` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.463321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] RBAC: Authorizing `identity:get_user(user_id=9f1f76c9bdb64c12856c29da8756eaa7)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.467097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.467097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 103/196] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/users/9f1f76c9bdb64c12856c29da8756eaa7 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:17.472729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.493852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 94/197] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:17 2022] POST /identity/v3/users => generated 293 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:17.513233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.591999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291d237-2c55-46ae-9bb3-09ed4f867ede None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.601857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.631133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53daeb2f-d54a-4414-888b-115b2693468f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.631918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.632580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.632862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.633137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.634344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.636089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.644103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 104/198] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:17.645314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.673386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.674601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.674913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.675178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.687612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:17.693793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:17.694731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:17.695525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 95/199] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:17.701402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.719096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:17.772690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dd934-c554-4c80-8aeb-484ba38c4906 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.815391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.816067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.816342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.816613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/9f1f76c9bdb64c12856c29da8756eaa7/roles/81ae57558f38464ebce24e926397f7e9` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.819382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, user_id=9f1f76c9bdb64c12856c29da8756eaa7, role_id=81ae57558f38464ebce24e926397f7e9)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.820980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.835707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358b47fe-8d29-49e8-ace3-c81452745778 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:17.875676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:17.876255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:17.876529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:17.876859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:17.878132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:17.878508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:17.879875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:17.889852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 96/200] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:17.895296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 105/201] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:17 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/users/9f1f76c9bdb64c12856c29da8756eaa7/roles/81ae57558f38464ebce24e926397f7e9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:17.898762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:17.985960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:18.019722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901adef-b632-4128-ace0-d70c6f321926 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:18.061946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:18.062995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:18.063477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:18.063971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] PATH_INFO: `/v3/users/99334971887945669c6f337a2abd686a` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:18.065471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=99334971887945669c6f337a2abd686a)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:18.067333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:18.069279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 97/202] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:17 2022] GET /identity/v3/users/99334971887945669c6f337a2abd686a => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:18.075853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:18.121143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:18.192712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e696bf4-ba64-4900-9715-da75546de999 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:18.231350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:18.232026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:18.232026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:18.232865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:18.303717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:18.305194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:18.339881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:18.410956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 106/203] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:18.416817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:18.535338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77862854-c4ac-429d-8f34-f4e39b9a8fc4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:18.575726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:18.576759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:18.577253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:18.577757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:18.579157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:18.581399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38324e-6610-4f66-8470-ea1f4bf73905 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:18.583717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 98/204] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:18 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:18.592021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:18.691586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fdcc5-9926-4a65-9300-9d726af56a66 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:18.726412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:18.727054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:18.727332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:18.727600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:18.728330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] RBAC: Adding query filter params (role.id=8a04c4ea718349e293dc17454cef7b86, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871, user.id=99334971887945669c6f337a2abd686a) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:18.728639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:18.730085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce72c412-9559-4ccc-8d03-550a118115f7 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:18.731889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:18.736590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 107/205] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:18 2022] GET /identity/v3/role_assignments?user.id=99334971887945669c6f337a2abd686a&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:19.873288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0653b2-af20-41c7-ae2e-d6a83aa0b661 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:19.873846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f4d94-a7de-4c7b-aebf-461516988a8e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:19.873846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f4d94-a7de-4c7b-aebf-461516988a8e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:19.874306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0653b2-af20-41c7-ae2e-d6a83aa0b661 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:19.874306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0653b2-af20-41c7-ae2e-d6a83aa0b661 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:19.874469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f4d94-a7de-4c7b-aebf-461516988a8e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:19.878672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 99/207] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:19.878802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 108/207] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:19.880936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669109b-f90e-4559-831b-ed4b2dcf0629 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:19.881221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669109b-f90e-4559-831b-ed4b2dcf0629 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:19.881499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669109b-f90e-4559-831b-ed4b2dcf0629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:19.883971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54053-bbd0-4f89-9bff-ef8d02675c19 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:19.884258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54053-bbd0-4f89-9bff-ef8d02675c19 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:19.884557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb54053-bbd0-4f89-9bff-ef8d02675c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:19.900824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d669109b-f90e-4559-831b-ed4b2dcf0629 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:19.902021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d669109b-f90e-4559-831b-ed4b2dcf0629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:19.903083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1eb54053-bbd0-4f89-9bff-ef8d02675c19 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:19.904239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1eb54053-bbd0-4f89-9bff-ef8d02675c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:19.905947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:19.973143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 100/208] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:19.978675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 109/209] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:20.226247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b897e-8429-459d-abc4-44a06ddfaa34 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.226646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b897e-8429-459d-abc4-44a06ddfaa34 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.226858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b897e-8429-459d-abc4-44a06ddfaa34 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.228317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 101/210] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:20.233218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.233503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.233776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.245793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b897e-8429-459d-abc4-44a06ddfaa34 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.252394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:20.253572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:20.319922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 110/211] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:20.320771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.349822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b897e-8429-459d-abc4-44a06ddfaa34 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.381759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:20.384163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.384800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.385080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.385355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.394734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:20.397962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:20.402786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:20.403649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:20.404431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 102/212] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:20.410354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.427048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb815cb-6d1a-461f-b4b7-7ca9ca2329be None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.462219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.462872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.463157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.463427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.464532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:20.465980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:20.467571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 111/213] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:20.473762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.517111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb1d31d-504d-4cfd-a3cc-dc19d8c1fcc6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.552078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.552730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.553008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.553273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.553928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:20.554224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:20.555627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:20.563720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 103/214] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:20.569838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.587151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f65295-519f-47df-ad69-7b53a9c53437 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.613714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:20.622357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.626348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.626348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.626348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.626348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:20.626348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:20.631659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:20.669675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625699db-b3f1-4d9b-a510-ed16ed89a011 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.702239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.702881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.703153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.703418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] PATH_INFO: `/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.716374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 112/215] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:20 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:45:20.717199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] RBAC: Authorizing `identity:get_user(user_id=427622f6e9bf4785b5a07ebabc1d8f2c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:20.720901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.721060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:20.721060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 104/216] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:20.726876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:20.844007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eaa669-34f2-4690-8a47-fe3cc48a1efc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.853761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc792e79-d9eb-48d0-9170-78befeb2b732 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:20.889186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:20.907045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.908022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.908473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.908842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:20.909749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.910184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] RBAC: Adding query filter params (role.id=8a04c4ea718349e293dc17454cef7b86, user.id=427622f6e9bf4785b5a07ebabc1d8f2c, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:20.910702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:20.910964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:20.911402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:20.913045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:20.913233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:20.921569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 105/217] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/role_assignments?user.id=427622f6e9bf4785b5a07ebabc1d8f2c&scope.system=all&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:20.928154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:20.933719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:20.935813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:20.936786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 113/218] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:20.944534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:21.065325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442e61-6652-4091-9e6a-97d80c8ef8c9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:21.107371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:21.108346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:21.108793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:21.109208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:21.110019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:21.110466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:21.112054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:21.121074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 106/219] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:21.127236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:21.174782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:21.238490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0c9e6-2e2d-4dcd-b92e-fc67b0d72190 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:21.273146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:21.273831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:21.274107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:21.274373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] PATH_INFO: `/v3/users/9f1f76c9bdb64c12856c29da8756eaa7` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:21.275592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] RBAC: Authorizing `identity:get_user(user_id=9f1f76c9bdb64c12856c29da8756eaa7)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:21.277001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:21.279116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 114/220] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:21 2022] GET /identity/v3/users/9f1f76c9bdb64c12856c29da8756eaa7 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:21.285548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:21.340945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:21.400778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92634884-a453-464f-b545-74880f45e6e4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:21.437224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:21.437837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:21.438112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:21.438380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:21.511050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:21.512600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:21.559882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:21.616570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 107/221] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:21.622023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:21.729709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa2a87-f2f0-4cae-ba76-292da1291f64 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:21.740882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:21.765797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:21.766470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:21.766777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:21.767048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:21.768911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:21.770344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a081a1-53fd-4d3d-a8e0-4a5c4e4dc9e3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:21.772439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 115/222] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:21 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:21.779303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:21.892487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8fbf37-0039-4cd3-b772-fe3d35ab178c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:21.928323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:21.928931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:21.929209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:21.929477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:21.930193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] RBAC: Adding query filter params (role.id=81ae57558f38464ebce24e926397f7e9, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871, user.id=9f1f76c9bdb64c12856c29da8756eaa7) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:21.930498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:21.931995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94345c40-1cf7-4a9e-b64a-0f4243aaf0db None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:21.937828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 108/223] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:21 2022] GET /identity/v3/role_assignments?user.id=9f1f76c9bdb64c12856c29da8756eaa7&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=81ae57558f38464ebce24e926397f7e9 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:23.112362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048e34-5526-4daa-a41c-1dba288e8b49 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.112864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048e34-5526-4daa-a41c-1dba288e8b49 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.113046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048e34-5526-4daa-a41c-1dba288e8b49 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.114240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 116/224] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:23.119507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee199b0-2485-4b59-8f62-58a851193ab3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.120236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee199b0-2485-4b59-8f62-58a851193ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.120651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee199b0-2485-4b59-8f62-58a851193ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.139329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-eee199b0-2485-4b59-8f62-58a851193ab3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:23.140760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-eee199b0-2485-4b59-8f62-58a851193ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:23.144933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:23.156693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c5c9-c687-4937-af74-2231e265a03b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.164170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c5c9-c687-4937-af74-2231e265a03b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.164170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3782c5c9-c687-4937-af74-2231e265a03b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.164170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 117/225] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:23.164416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.164667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.164941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.183838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:23.185021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:23.210047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 109/226] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:23.211327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d757-4560-4c37-a322-03385fba8db6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.211619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d757-4560-4c37-a322-03385fba8db6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.211892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432d757-4560-4c37-a322-03385fba8db6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.213308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 110/227] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:23.218333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.218621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.218922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.238181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:23.239318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:23.254249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 118/228] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:23.309930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 111/229] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:23.480737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.526509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.579476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b814e634-a1a7-4862-aff0-032edd12e30c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.589941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:23.600501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:23.614657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.615375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.615645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.615911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.625794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:23.631028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9262693-8bd0-406b-8f3e-67c812e16c12 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:23.631475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:23.632363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:23.633144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 119/230] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:23.634018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.669476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.670099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.670371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.670656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.681044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:23.685764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:23.686612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:23.687428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 112/231] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:23.689828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.732766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5958e-eaaf-4e3a-be40-a52cf4790604 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.768201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.768819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.769093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.769358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.770622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:23.772342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:23.773872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 120/232] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:23.774621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.788949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e0c90-6c3d-4162-95e0-c7da3d90e773 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.809013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:23.817252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:23.823236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.823841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.824121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.824391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.825051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:23.825355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:23.826863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:23.836153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 113/233] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:23.836945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.875914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb017e4f-631a-4d16-b3c8-59b7f10010d4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.910085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.910742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.911017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.911283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.911940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:23.912250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:23.913600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:23.921335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 121/234] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:23.922169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:23.935737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7552613-6851-4578-9c3e-2644c7a8249c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:23.969125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:23.969729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:23.970004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:23.970275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] PATH_INFO: `/v3/groups` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:23.971898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:23.973312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:24.024226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:24.027965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a31b5b-0ff1-41ee-b269-c733fa87d499 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:24.049399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 114/235] 10.222.0.60 () {60 vars in 1134 bytes} [Fri Oct 21 14:45:23 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:24.049399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:24.066170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:24.066865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:24.067155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:24.067651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] PATH_INFO: `/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:24.069337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] RBAC: Authorizing `identity:get_user(user_id=427622f6e9bf4785b5a07ebabc1d8f2c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:24.070874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:24.072571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 122/236] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:23 2022] GET /identity/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:24.078485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:24.117141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:24.170958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb51a5f5-c2a9-4cd4-a8b4-e1d59e0e1a04 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:24.195398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4d642c-b04c-4fe2-a6f9-552b195ccd8c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:24.212166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:24.212923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:24.213266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:24.213648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] PATH_INFO: `/v3/users/615c85472ddb41e4a3d00bea0f6c619f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:24.230717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] RBAC: Authorizing `identity:get_user(user_id=615c85472ddb41e4a3d00bea0f6c619f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:24.232198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:24.233907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 115/237] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:24 2022] GET /identity/v3/users/615c85472ddb41e4a3d00bea0f6c619f => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:24.238236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:24.239060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:24.240031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:24.240724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:24.241097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] PATH_INFO: `/v3/system/users/427622f6e9bf4785b5a07ebabc1d8f2c/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:24.243881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=427622f6e9bf4785b5a07ebabc1d8f2c, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:24.244534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc680-ae9d-4187-8bc3-f911b70692f0 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:24.329494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 123/238] 10.222.0.60 () {58 vars in 1334 bytes} [Fri Oct 21 14:45:24 2022] PUT /identity/v3/system/users/427622f6e9bf4785b5a07ebabc1d8f2c/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:24.346600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606bea9-3d29-4e2c-b26f-acba1516703a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:24.360140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:24.385695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:24.386328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:24.386676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:24.386935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:24.387724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] RBAC: Adding query filter params (role.id=81ae57558f38464ebce24e926397f7e9, user.id=615c85472ddb41e4a3d00bea0f6c619f, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:24.388039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:24.389590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e93278f-b74a-43d7-8350-4e9873916dc7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:24.396640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 116/239] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Oct 21 14:45:24 2022] GET /identity/v3/role_assignments?user.id=615c85472ddb41e4a3d00bea0f6c619f&scope.system=all&role.id=81ae57558f38464ebce24e926397f7e9 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:26.251391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53899-33ab-4c0a-8fdc-7fb382c07298 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.252419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53899-33ab-4c0a-8fdc-7fb382c07298 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.252496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f53899-33ab-4c0a-8fdc-7fb382c07298 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.253853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 124/240] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:26.258976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61d11-8ce2-4279-8884-11a218a7c6c0 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.259348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61d11-8ce2-4279-8884-11a218a7c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.259516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d61d11-8ce2-4279-8884-11a218a7c6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.278363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-38d61d11-8ce2-4279-8884-11a218a7c6c0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:26.279581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-38d61d11-8ce2-4279-8884-11a218a7c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:26.349752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 117/241] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:26.499384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5eb4f-051c-46b5-bb0f-dcf296df1a26 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.499674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5eb4f-051c-46b5-bb0f-dcf296df1a26 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.500153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5eb4f-051c-46b5-bb0f-dcf296df1a26 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.501860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 125/242] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:26.507132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.507490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.507836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.526624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:26.527820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:26.578219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9644a-f004-436d-b276-51ca36d5d1c2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.578507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9644a-f004-436d-b276-51ca36d5d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.578825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9644a-f004-436d-b276-51ca36d5d1c2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.580875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 126/243] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:26.585529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.585812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.586092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.598677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 118/244] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:26.604794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:26.606003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:26.627527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:26.631576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:26.662532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:26.684015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 127/245] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:26.729165 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329abf2e-ba43-49e2-9876-2caa7408492a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:26.762658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:26.763234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.763510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.763780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.764877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:26.766309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:26.768672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 119/246] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:26.774247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:26.872188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a76b-3280-4b0e-80c9-074aa809e464 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:26.878772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:26.911763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:26.912380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:26.912658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:26.912937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] PATH_INFO: `/v3/groups` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:26.913576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:26.915067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:26.982504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a085-f91a-4bc2-aca8-484fae7ff2b5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:26.984070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.016883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.018249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.018531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.018832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.020501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 128/247] 10.222.0.60 () {60 vars in 1134 bytes} [Fri Oct 21 14:45:26 2022] POST /identity/v3/groups => generated 226 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:27.022426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.030845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:27.036952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:27.038158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:27.039938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 120/248] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:27.044530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.054654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.144791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821223e-062d-4704-963b-b447a7daf872 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.168379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a071c8ca-b940-48d5-bcb5-723b22f6f476 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.183425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.184074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.184356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.184628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.196832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:27.202553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:27.203464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:27.204236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 129/249] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:27.209742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.209866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.210390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.210711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.210989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.211738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:27.212058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.213938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.222732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 121/250] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:27.228424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.260877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.305780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.328510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95663734-8e0f-4bfe-83b4-0d39de0d9524 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.360199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7227a5-3a9e-42ea-ad98-efb0cc5a1f9e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.377238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.378011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.380822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.380822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.380822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:27.380822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.381817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.398733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 130/251] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:27.400886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.402859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.403553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.404032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.404314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] PATH_INFO: `/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.406344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] RBAC: Authorizing `identity:get_user(user_id=427622f6e9bf4785b5a07ebabc1d8f2c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.408580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.410035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 122/252] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/users/427622f6e9bf4785b5a07ebabc1d8f2c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:27.415856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.527250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609130e2-34bc-4325-b586-b3d015c7bf24 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.548015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.569498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089b9cc-a273-47e2-afe7-a20631ec473d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.570539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.571268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.571554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.571829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] PATH_INFO: `/v3/users/615c85472ddb41e4a3d00bea0f6c619f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.573118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] RBAC: Authorizing `identity:get_user(user_id=615c85472ddb41e4a3d00bea0f6c619f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.574432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.576500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 131/253] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/users/615c85472ddb41e4a3d00bea0f6c619f => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:27.581790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:27.598739 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:27.621180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.621793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.622070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.622334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.623144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] RBAC: Adding query filter params (role.id=8a04c4ea718349e293dc17454cef7b86, user.id=427622f6e9bf4785b5a07ebabc1d8f2c, scope.system=all) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:27.623414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.624863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c683e-400f-4454-b605-f3e20f68e51a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.642954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 123/254] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Oct 21 14:45:27 2022] GET /identity/v3/role_assignments?user.id=427622f6e9bf4785b5a07ebabc1d8f2c&scope.system=all&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 510 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:27.715996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9626dd98-c262-4ccc-9d0b-fa4aecd08f65 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:27.754624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:27.755317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:27.755602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:27.755878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] PATH_INFO: `/v3/system/users/615c85472ddb41e4a3d00bea0f6c619f/roles/81ae57558f38464ebce24e926397f7e9` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:27.757481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=615c85472ddb41e4a3d00bea0f6c619f, role_id=81ae57558f38464ebce24e926397f7e9)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:27.758907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3ce00e-939f-4501-a76d-79c25ff3e520 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:27.835002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 132/255] 10.222.0.60 () {58 vars in 1334 bytes} [Fri Oct 21 14:45:27 2022] PUT /identity/v3/system/users/615c85472ddb41e4a3d00bea0f6c619f/roles/81ae57558f38464ebce24e926397f7e9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:29.477281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21d15-2028-4153-804d-7365ead98acd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:29.478281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21d15-2028-4153-804d-7365ead98acd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:29.478822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae21d15-2028-4153-804d-7365ead98acd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:29.480674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 124/256] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:29.486922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:29.487772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:29.488263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:29.508709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:29.510248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:29.584252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 133/257] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 14:45:30.100132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae21d15-2028-4153-804d-7365ead98acd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:30.116288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae21d15-2028-4153-804d-7365ead98acd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:30.120803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:30.121437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.121718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.121987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.139576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:30.145133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:30.146005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:30.146813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 125/258] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:45:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:30.152615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:30.183174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821cd3d-fc99-4716-a0a8-fe5e71221295 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:30.188244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:30.188871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.190267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.190267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.191041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:30.191347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:30.193155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:30.193155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 14:45:30.193155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 14:45:30.194069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:30.212248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 134/259] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:45:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:30.218237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:30.233146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a3fc7-8e45-409b-bcec-4e20929639bf admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:30.244987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:30.245644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.245919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.246187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.257135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:30.258971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:30.258971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 14:45:30.258971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 14:45:30.260199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:30.261876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 126/260] 10.222.0.60 () {56 vars in 1201 bytes} [Fri Oct 21 14:45:30 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:30.268503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:30.281247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b361f53c-68aa-42ce-a599-e186ab3b38d2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:30.288459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:30.289164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.289453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.289733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.290998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:30.293070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe501a1-d589-40e3-899a-140bd2b61762 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:30.294928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 135/261] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:45:30 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:30.301563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:30.316667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c304ec-57d1-4b8e-859c-1bba0605a377 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:30.326773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:30.327773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.329405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.329706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/groups/2df31ec3687a42c7ab441da301807530/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.335958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, group_id=2df31ec3687a42c7ab441da301807530, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:30.337001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:30.337001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 14:45:30.337001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 14:45:30.338978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f096179a-5536-43e7-91fc-b61f0b4f5c65 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:30.398136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835185d2-0dfe-4e75-9838-0add6290922e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.398451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835185d2-0dfe-4e75-9838-0add6290922e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.398844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835185d2-0dfe-4e75-9838-0add6290922e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.400262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 136/262] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:30.405364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6818-f26d-4ad4-863f-3a27bcb4dcda None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.405648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6818-f26d-4ad4-863f-3a27bcb4dcda None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.405930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f6818-f26d-4ad4-863f-3a27bcb4dcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.425417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:30.430612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 127/263] 10.222.0.60 () {58 vars in 1463 bytes} [Fri Oct 21 14:45:30 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/groups/2df31ec3687a42c7ab441da301807530/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:30.433061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f04f6818-f26d-4ad4-863f-3a27bcb4dcda None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:30.437285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f04f6818-f26d-4ad4-863f-3a27bcb4dcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:30.522363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 137/264] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 14:45:30.717200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f9419-b09a-4200-b822-1114f215db6d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.717200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f9419-b09a-4200-b822-1114f215db6d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.717200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f9419-b09a-4200-b822-1114f215db6d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.718244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 128/265] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:30.724763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196692b-c060-4b01-8249-4e79fc8b5557 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.725359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196692b-c060-4b01-8249-4e79fc8b5557 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.725824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196692b-c060-4b01-8249-4e79fc8b5557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.749102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5196692b-c060-4b01-8249-4e79fc8b5557 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:30.750686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5196692b-c060-4b01-8249-4e79fc8b5557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:30.765315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 138/266] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 21 14:45:30.779946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a6f2-8d31-4788-981f-1c13ec9ed294 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.780521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a6f2-8d31-4788-981f-1c13ec9ed294 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.780984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1854a6f2-8d31-4788-981f-1c13ec9ed294 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.782805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 129/267] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:30.788393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:30.788672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:30.788949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:30.815512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:30.820766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:30.892162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 139/268] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:30.987483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a6f2-8d31-4788-981f-1c13ec9ed294 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.003862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1854a6f2-8d31-4788-981f-1c13ec9ed294 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.008637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.009515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.009983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.010460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.031649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:31.037983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:31.038877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:31.039761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:31.041995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 130/269] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:45:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:31.047907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.065712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40db66d-9ad0-4230-aaf0-9b182f189596 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.073901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.074528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.074861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.075137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.075819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:31.076122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.077741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.090149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 140/270] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:31.097168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.110755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.113314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca93e5d-b5c3-4c29-956f-f4114a89dbb1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.124579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7058412-21a3-4c9c-a757-258414d88211 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.127251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.127913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.128190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.128459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.129976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.131935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.133691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 131/271] 10.222.0.60 () {56 vars in 1201 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:31.139510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.140461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.141124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.141405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.141673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.150340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a176fc3-9570-41fa-b6fc-e657941ba8e6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.162809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.163455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.163737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.164193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.165612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.167798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.169455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 132/272] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:31.176791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:31.176932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:31.182209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:31.183125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:31.183913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 141/273] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 73 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:31.189980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.193551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6534d53a-3daa-424a-b343-23faafb7b294 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.203284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.204052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.204248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.204526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/groups/2df31ec3687a42c7ab441da301807530/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.206699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc46fb03-facb-4e90-a1f6-a1a9541afa7a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.211587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bec480032c6457f9f1d21ddc43f7f79, group_id=2df31ec3687a42c7ab441da301807530, role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.214369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.216606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.217216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.217492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.217765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.218432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] RBAC: Adding query filter params (name=member) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:31.218764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.221198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.238485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 142/274] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:31.244050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.259911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe04aae-bdca-4638-bc50-aa64002c717b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.270979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.271608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.271886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.272405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.279315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.280352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:31.280352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 14:45:31.280352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 14:45:31.281553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.284351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 143/275] 10.222.0.60 () {56 vars in 1201 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:31.289575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.298245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 133/276] 10.222.0.60 () {58 vars in 1463 bytes} [Fri Oct 21 14:45:31 2022] PUT /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79/groups/2df31ec3687a42c7ab441da301807530/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:31.304386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec78f3e-8557-435b-98c9-b3f1420fb7bf admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.348588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.367020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:31.390340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.391027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.391305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.391576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.395487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.397862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.399974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 144/277] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:31.405835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.420286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef77df1-4f9a-4296-b6a6-412f36f77d90 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.441099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.441786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.442069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.442345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/groups/2df31ec3687a42c7ab441da301807530/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.448123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, group_id=2df31ec3687a42c7ab441da301807530, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.449178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:31.449178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 14:45:31.449178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 14:45:31.451204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.549467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:31.567356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f5bb3e-ff0f-4b78-8fa2-ec388bdac4f8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.576681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 145/278] 10.222.0.60 () {58 vars in 1463 bytes} [Fri Oct 21 14:45:31 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/groups/2df31ec3687a42c7ab441da301807530/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:31.610584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.612304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.612856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.613338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.629481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d182e7d7-56e8-4645-ae0b-5f89d06ca5bf None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:31.634856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:31.635773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 21 14:45:31.636555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:31.639318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 134/279] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:31.645614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:31.678233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875584-ace9-47d7-b5a3-2b786ff964f4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.679383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875584-ace9-47d7-b5a3-2b786ff964f4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.679865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26875584-ace9-47d7-b5a3-2b786ff964f4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.681509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 135/280] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:31.687225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.687802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.688306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.719443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:31.722081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:31.838728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 136/281] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 14:45:31.866097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97774f42-b5f6-4b56-b5e6-8b7af1a87b25 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:31.910569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:31.911217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:31.911500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:31.911778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:31.912454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:31.912764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:31.914155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:31.926680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 146/282] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:31.932534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.070028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:32.079304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e020bf5-f20c-4e58-8255-b2ef9de9c339 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.128000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.128567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.128842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.129111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] PATH_INFO: `/v3/users/615c85472ddb41e4a3d00bea0f6c619f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.135314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] RBAC: Authorizing `identity:get_user(user_id=615c85472ddb41e4a3d00bea0f6c619f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.136790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.138725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 137/283] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:45:31 2022] GET /identity/v3/users/615c85472ddb41e4a3d00bea0f6c619f => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:32.144646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.188849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:32.256145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e5358f-070d-4afc-9470-128f1e33879c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.301582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.302203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.302483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.302781 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.303520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] RBAC: Adding query filter params (role.id=81ae57558f38464ebce24e926397f7e9, user.id=615c85472ddb41e4a3d00bea0f6c619f, scope.system=all) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:32.303830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.305281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.322245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 147/284] 10.222.0.60 () {56 vars in 1308 bytes} [Fri Oct 21 14:45:32 2022] GET /identity/v3/role_assignments?user.id=615c85472ddb41e4a3d00bea0f6c619f&scope.system=all&role.id=81ae57558f38464ebce24e926397f7e9 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:32.339931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.356520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440596cf-2859-4596-a690-b71f49b347c6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.364918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.365567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.365863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.366118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.389619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:32.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:32.395545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:32.396338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 138/285] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:45:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 56 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:32.403936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.421649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92503e0-81ef-4513-bd34-e4a27b4f3a36 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.428740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.429382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.429664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.429962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.430690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:32.431015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.432692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.454446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 148/286] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Oct 21 14:45:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:32.460304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.484146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82b6766-3360-4546-97f5-7a2b2f73b71f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.491228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.492333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.492874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.493342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.494943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.497160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.499242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 139/287] 10.222.0.60 () {56 vars in 1201 bytes} [Fri Oct 21 14:45:32 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:32.505240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.521129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c894605b-c78a-4f29-a948-b23d0b611eb7 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.530949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.531607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.531893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.532169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.533961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.536094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa007a1-8155-4890-84e3-a60096e21200 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.538048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 149/288] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:45:32 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:32.543831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:32.558660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e0a9ee-5573-47ea-8258-6104b789ab5f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:32.567145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:32.568094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.568593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.569136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/groups/2df31ec3687a42c7ab441da301807530/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.571575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f678e9b6ffe4e7ebc7457f71cca3871, group_id=2df31ec3687a42c7ab441da301807530, role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:32.574829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cece9d6-e781-434a-9f79-645eff86ad6c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:32.702772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 140/289] 10.222.0.60 () {58 vars in 1463 bytes} [Fri Oct 21 14:45:32 2022] PUT /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871/groups/2df31ec3687a42c7ab441da301807530/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:32.845843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7bd95-f464-4e97-9a02-f6f852f1234a None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.846399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7bd95-f464-4e97-9a02-f6f852f1234a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.846399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed7bd95-f464-4e97-9a02-f6f852f1234a None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.847793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 150/290] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:32.853804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:32.854776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:32.855325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:32.885780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:32.887648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:32.972464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 141/291] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 21 14:45:33.358912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7bd95-f464-4e97-9a02-f6f852f1234a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:33.373072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed7bd95-f464-4e97-9a02-f6f852f1234a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:33.380162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:33.380809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.381088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.381360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.393253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:33.398831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:33.399715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:33.401062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 151/292] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:45:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:33.405753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:33.422667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45719d4-55c3-4f9d-93c8-9764b4362f66 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:33.432200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:33.433111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.433590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.434155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.435064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:33.435567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:33.436772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:45:33.436772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 14:45:33.436772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 14:45:33.438000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:33.456145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 142/293] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:45:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:33.461748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:33.476347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92dcf7-2575-4b77-abce-f94e36c4d81f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:33.487129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:33.487790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.488086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.488352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] PATH_INFO: `/v3/groups/cb03d7208610447498384b5595954653` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.499213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] RBAC: Authorizing `identity:get_group(group_id=cb03d7208610447498384b5595954653)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:33.501151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:33.503767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 152/294] 10.222.0.60 () {56 vars in 1201 bytes} [Fri Oct 21 14:45:33 2022] GET /identity/v3/groups/cb03d7208610447498384b5595954653 => generated 229 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:33.509070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:33.520372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514e5cc-ea69-4312-9ccf-a1749a605cea admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:33.530298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:33.531248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.531729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.532238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] PATH_INFO: `/v3/projects/1cd93ea5b05642848d07178b50fbac9f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.533640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] RBAC: Authorizing `identity:get_project(project_id=1cd93ea5b05642848d07178b50fbac9f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:33.535673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db279bea-4e71-44f5-8455-e6768c8e1e21 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:33.537539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 143/295] 10.222.0.60 () {56 vars in 1207 bytes} [Fri Oct 21 14:45:33 2022] GET /identity/v3/projects/1cd93ea5b05642848d07178b50fbac9f => generated 344 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:33.543420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:33.559552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33c94e-ad2e-4c8f-a8b4-8a416fcbf7d6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:33.567301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:33.567984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.568268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.568547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] PATH_INFO: `/v3/projects/1cd93ea5b05642848d07178b50fbac9f/groups/cb03d7208610447498384b5595954653/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.570478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd93ea5b05642848d07178b50fbac9f, group_id=cb03d7208610447498384b5595954653, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:33.573185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b287d-c1aa-45d6-bf64-282a40098952 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:33.662872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 153/296] 10.222.0.60 () {58 vars in 1463 bytes} [Fri Oct 21 14:45:33 2022] PUT /identity/v3/projects/1cd93ea5b05642848d07178b50fbac9f/groups/cb03d7208610447498384b5595954653/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:33.986655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456550f7-481f-42f4-a97d-ce4e9c8a9ae0 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.987558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456550f7-481f-42f4-a97d-ce4e9c8a9ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.988040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456550f7-481f-42f4-a97d-ce4e9c8a9ae0 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:33.989962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 144/297] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:33.995249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:33.995713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:33.995786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.021199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:34.022681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:34.077412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:34.102602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 154/298] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:34.469190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e11d2-3ec4-4ea0-bc2a-929163140207 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.469870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e11d2-3ec4-4ea0-bc2a-929163140207 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.470432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2e11d2-3ec4-4ea0-bc2a-929163140207 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.472054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 145/299] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:34.480346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd413-55c1-4fca-9263-3423a46041b6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.480913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd413-55c1-4fca-9263-3423a46041b6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.481390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bd413-55c1-4fca-9263-3423a46041b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.486490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:34.505400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-eb1bd413-55c1-4fca-9263-3423a46041b6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:34.507088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-eb1bd413-55c1-4fca-9263-3423a46041b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:34.576102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:34.591898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 146/300] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:34.593189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b56-95e5-493e-bb59-d23bf1a3edac None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.593475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b56-95e5-493e-bb59-d23bf1a3edac None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.593753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a67b56-95e5-493e-bb59-d23bf1a3edac None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.599781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 147/301] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:34.603572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.612280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.612588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.635556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:34.637094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:34.660910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3bfe-be7c-4f15-8b5d-18c1579c157e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:34.715318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 148/302] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:34.744810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:34.745385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.745667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.745935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.766542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:34.772010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:34.773152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:34.773934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 155/303] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:34.779502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:34.933910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cab5eb-3c7e-440c-b4dc-ced95809e6bd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:34.964982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:34.973991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:34.992068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:34.992687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:34.992940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:34.993207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:34.993869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:34.994171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:34.995619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.005254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 149/304] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:35.010939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.059796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.127059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6a6f4-a149-4275-81b4-4ce77f5c0648 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.148641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac2d5d-4d19-4816-a004-2f2e75c367b2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.178174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.179882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.180171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.180443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.195705 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:35.200751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:35.201621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:35.202440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 156/305] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:35.203136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.211570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.215729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.215729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.215922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.283420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.301023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:35.303398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.330771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.334794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2720f4-2195-4c39-bbeb-3a23c3b26bb3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.373875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.374498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.374850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.375122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.388247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:35.394883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:35.395870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:35.396660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 157/306] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:35.402085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.436117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 150/307] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 21 14:45:35.437686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.533526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84de79a-593a-4ce6-bc63-e6a61fbf8a3f None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.601318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.601318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a9fb6-5771-4583-8b32-48924b45b531 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.601556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.601556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.601556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.601556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.601843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] RBAC: Adding query filter params (name=member) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:35.601843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:35.601843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.609169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 158/308] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:35.613771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.629141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.630142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.630428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.630732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.631412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:35.631719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:35.633270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.647951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 151/309] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:35.647951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.653156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.764799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5dd43c-3937-45f7-9bb0-478f9e6fa224 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.794170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d19392-cd52-49e1-9345-43b6391845dd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.817782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.818884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.819384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.820945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.822377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:35.824034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.827047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 159/310] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:35.828005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:35.851222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83673da-092f-4a43-9552-86b838349370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.852126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83673da-092f-4a43-9552-86b838349370 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.852579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83673da-092f-4a43-9552-86b838349370 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.853098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83673da-092f-4a43-9552-86b838349370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:35.873686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.930077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:35.933039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83673da-092f-4a43-9552-86b838349370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:35.934505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83673da-092f-4a43-9552-86b838349370 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:35.942435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ef7d4e-a4ba-4c42-85cf-c8d2df062af3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:35.991266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:35.991886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:35.992557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:35.992832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.060716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 152/311] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:36.061586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83673da-092f-4a43-9552-86b838349370 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.089186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.092161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7386451-b02c-4d38-86d1-152c4e252a23 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.131750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.205342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83673da-092f-4a43-9552-86b838349370 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.208355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.218915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 160/312] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:36.228954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ddcd-3c97-438e-9bc0-29e60e45d190 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.228954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ddcd-3c97-438e-9bc0-29e60e45d190 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.228954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa48ddcd-3c97-438e-9bc0-29e60e45d190 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.228954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 161/313] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:36.228954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ddcd-3c97-438e-9bc0-29e60e45d190 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.260775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.261828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.262422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.262734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.264037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.265966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.268052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 153/314] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.273797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.367384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa48ddcd-3c97-438e-9bc0-29e60e45d190 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.404559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a18f7-7799-4c61-bc6f-44814f29abbe None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.415077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.415790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.415933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.416175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.418800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.419459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e75609-38a7-4d5e-a38d-570dce0b40b2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.421314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 162/315] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.424019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.424669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.424669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.429934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.444281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:36.445562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:36.450123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.450950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.451296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.451573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] PATH_INFO: `/v3/projects/9bec480032c6457f9f1d21ddc43f7f79` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.452893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] RBAC: Authorizing `identity:get_project(project_id=9bec480032c6457f9f1d21ddc43f7f79)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.454393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.457896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 154/316] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/projects/9bec480032c6457f9f1d21ddc43f7f79 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.459237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.521993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.539933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 163/317] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:36.541356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.588880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c6fb84-97a8-471f-9465-f3334a3b63fc None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.634792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.634792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.634792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.634792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.635228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] RBAC: Adding query filter params (group.id=2df31ec3687a42c7ab441da301807530, role.id=8a04c4ea718349e293dc17454cef7b86, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:36.635228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.635228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.640596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 155/318] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/role_assignments?group.id=2df31ec3687a42c7ab441da301807530&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.641408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.674157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b13bd3-36e2-4dd3-bd48-0bf96899cb64 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.706166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.720295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.721274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.721727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.722155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.723507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.725122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c53c95-37e8-467c-a2bc-a292df6a2230 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.726780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 164/319] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.728007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00bc5-d7bd-4437-84fb-37b5936e9cd2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.728359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00bc5-d7bd-4437-84fb-37b5936e9cd2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.728629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc00bc5-d7bd-4437-84fb-37b5936e9cd2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.734221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 165/320] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:36.735821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc00bc5-d7bd-4437-84fb-37b5936e9cd2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.771854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdadbf2-2bb9-4d5c-93ec-ad6c825a432f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.818755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:36.822936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.823985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.824331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.824679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.825963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.827577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9f73d-02b3-4da4-9131-194ee77bf70e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.829732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 156/321] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:36.835578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.844029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.844029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.858660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:36.861849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:36.890538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc00bc5-d7bd-4437-84fb-37b5936e9cd2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:36.941279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:36.941690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:36.941973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:36.942243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:36.944408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] RBAC: Adding query filter params (group.id=2df31ec3687a42c7ab441da301807530, role.id=8a04c4ea718349e293dc17454cef7b86, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:36.944408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:36.946255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:36.950753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 157/322] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:36.951788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:36.958713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 166/323] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/role_assignments?group.id=2df31ec3687a42c7ab441da301807530&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=8a04c4ea718349e293dc17454cef7b86 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:36.958713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.040270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.108470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da67424-c197-4717-a225-0b6bc6dd51ec None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.117167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe19d59-ceae-4244-9946-7cfba5437b37 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.152121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.166576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.167333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.167437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.167616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.167893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.168046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.168329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.168591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.170020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] RBAC: Adding query filter params (group.id=2df31ec3687a42c7ab441da301807530, role.id=e6c31a1ad1c74840b2b04e21d9ff67bf, scope.project.id=9bec480032c6457f9f1d21ddc43f7f79) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:37.170330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.171846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.192959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 158/324] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/role_assignments?group.id=2df31ec3687a42c7ab441da301807530&scope.project.id=9bec480032c6457f9f1d21ddc43f7f79&role.id=e6c31a1ad1c74840b2b04e21d9ff67bf => generated 611 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:37.193084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:37.193865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:37.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 167/325] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:37.197321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.282715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.346095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649ecf8-dd21-44a3-875b-2f0bd22570ea None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.362424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.393768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.394387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.394683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.394955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.395616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:37.395915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.397527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.409519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 159/326] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:37.415350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.432699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bc7fd-467b-49cb-baba-500cde4bfe10 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.476246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.476915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.477196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.477465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.490048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:37.495496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:37.496359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:37.497155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 168/327] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:37.503358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.531033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ef5f9a-409e-409f-a71d-e8d28a75b80e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.555950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.568706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.570818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.570818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.570818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.613184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.629087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac841c0-ee2d-4431-a789-e6a618168dd1 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.646737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.649196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.670331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.671002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.671276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.671544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.672202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:37.672500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.675120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.684269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 169/328] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:37.690711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.769093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 160/329] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:37.774239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:37.809193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f7fa2-18ae-4d16-a65b-e7c0edcf4177 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.810780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.842640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.843328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.843609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.843877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.847348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:37.874704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39a072-7e9d-41cc-91f3-5bf57a240f6b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:37.908853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:37.909479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:37.909756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:37.910024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] PATH_INFO: `/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:37.911195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] RBAC: Authorizing `identity:get_project(project_id=6f678e9b6ffe4e7ebc7457f71cca3871)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.911772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:37.912708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.913337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:37.918775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 161/330] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/projects/6f678e9b6ffe4e7ebc7457f71cca3871 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:37.919990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:38.014018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 170/331] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 21 14:45:38.020361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:38.022476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:38.023471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02262ef5-c2c5-4b63-b374-6207d75abc00 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:38.056932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:38.057555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:38.057831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:38.058101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] PATH_INFO: `/v3/groups/2df31ec3687a42c7ab441da301807530` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:38.059212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] RBAC: Authorizing `identity:get_group(group_id=2df31ec3687a42c7ab441da301807530)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:38.060651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:38.062202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 162/332] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Oct 21 14:45:37 2022] GET /identity/v3/groups/2df31ec3687a42c7ab441da301807530 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:38.068643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:38.079168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:38.123498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da804cf-17d4-4653-a9b1-d9b617b27152 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:38.158332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:38.158979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:38.159257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:38.159519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] PATH_INFO: `/v3/projects/1cd93ea5b05642848d07178b50fbac9f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:38.160627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] RBAC: Authorizing `identity:get_project(project_id=1cd93ea5b05642848d07178b50fbac9f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:38.162037 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:38.163724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 171/333] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:45:38 2022] GET /identity/v3/projects/1cd93ea5b05642848d07178b50fbac9f => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:38.168827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d144007-2b11-4737-92a8-26566bbb20c1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:38.169268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:38.203584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:38.204188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:38.204470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:38.205362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:38.206086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] RBAC: Adding query filter params (group.id=2df31ec3687a42c7ab441da301807530, role.id=e6c31a1ad1c74840b2b04e21d9ff67bf, scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:38.206394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:38.207866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:38.213672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 163/334] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Oct 21 14:45:38 2022] GET /identity/v3/role_assignments?group.id=2df31ec3687a42c7ab441da301807530&scope.project.id=6f678e9b6ffe4e7ebc7457f71cca3871&role.id=e6c31a1ad1c74840b2b04e21d9ff67bf => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:38.239921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:38.273089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f05c78-ba9a-4062-b12d-782d5df918b4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:38.310417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:38.311098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:38.311380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:38.311654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] PATH_INFO: `/v3/groups/cb03d7208610447498384b5595954653` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:38.312807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] RBAC: Authorizing `identity:get_group(group_id=cb03d7208610447498384b5595954653)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:38.315173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45654a-51ab-4981-bf67-cf0a5d2473ea None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:38.316878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 172/335] 10.222.0.60 () {56 vars in 1180 bytes} [Fri Oct 21 14:45:38 2022] GET /identity/v3/groups/cb03d7208610447498384b5595954653 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:38.322913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:38.396661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:38.427120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec26a611-cac5-486c-9c8d-2c68669d89f6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:38.461445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:38.462052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:38.462335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:38.462606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:38.463387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] RBAC: Adding query filter params (group.id=cb03d7208610447498384b5595954653, role.id=55a557362b9542629b4209da47492ebf, scope.project.id=1cd93ea5b05642848d07178b50fbac9f) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:38.463698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:38.465145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf154b5-f5b6-40b0-9e57-d3f88d50cdcc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:38.471524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 164/336] 10.222.0.60 () {56 vars in 1376 bytes} [Fri Oct 21 14:45:38 2022] GET /identity/v3/role_assignments?group.id=cb03d7208610447498384b5595954653&scope.project.id=1cd93ea5b05642848d07178b50fbac9f&role.id=55a557362b9542629b4209da47492ebf => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:41.202582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1948-4b88-458b-beec-682814a963a5 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.203487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1948-4b88-458b-beec-682814a963a5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.203984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4b1948-4b88-458b-beec-682814a963a5 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.206201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 173/337] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:41.211558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b446-d53c-4d7b-85ad-5b0336d86ce3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.212406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b446-d53c-4d7b-85ad-5b0336d86ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.212827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629b446-d53c-4d7b-85ad-5b0336d86ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.239744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1629b446-d53c-4d7b-85ad-5b0336d86ce3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:41.241186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1629b446-d53c-4d7b-85ad-5b0336d86ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:41.336064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 165/338] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:41.388999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe05f83-8ee0-4c8e-a956-fcf68ab3e30b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.389633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe05f83-8ee0-4c8e-a956-fcf68ab3e30b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.390107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe05f83-8ee0-4c8e-a956-fcf68ab3e30b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.392438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 174/339] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:41.397820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.398106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.398378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.424905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:41.426090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:41.451812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc489ca1-aa26-45fe-9607-74eea4791f45 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.452892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc489ca1-aa26-45fe-9607-74eea4791f45 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.453387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc489ca1-aa26-45fe-9607-74eea4791f45 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.455782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 175/340] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:41.461439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.461990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.462362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.499469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:41.500583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:41.513898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:41.525587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 166/341] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:41.598449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 176/342] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:41.727799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:41.854612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79a9d12-9e8f-438f-9372-e3e32d64ebc9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:41.905856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:41.906703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:41.906987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:41.907262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:41.922925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:41.925006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:41.943837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:41.944825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:41.945671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:41.945671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:41.945671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:41.945671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 167/343] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:41.949294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:41.955919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.022874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.033281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.069986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd06776-34e4-45d4-bab6-8c7cfbeaf097 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.076480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21316b-5a1a-4fce-a811-c42a4242c64f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.105578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.106139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.106590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.106987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.111663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.112269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.112547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.112873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.113519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:42.113821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.115226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.117796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.119289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.133944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 168/344] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:42.135196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-0ec190ea-dee8-45a6-90d6-bbde5496b662 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:42.135345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:42.136280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:42.137120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:42.137120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:42.137120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:42.138874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 177/345] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:42.140245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e419008-a9e1-4783-bbfe-77ca52b67d46 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.140711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e419008-a9e1-4783-bbfe-77ca52b67d46 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.141151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e419008-a9e1-4783-bbfe-77ca52b67d46 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.142576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 178/346] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:42.143465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e419008-a9e1-4783-bbfe-77ca52b67d46 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.245079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2c6fde-6f8e-45cb-b73a-4e37607e9204 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.253777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e419008-a9e1-4783-bbfe-77ca52b67d46 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.255555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.277177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.281012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.281648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.281927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.282197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.291543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.293984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.293984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.293984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.293984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.294242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.294490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b664395-cc32-4980-b884-f9cc6d5ef34f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.294988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:42.310294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:42.311508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:42.312561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:42.312561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:42.312561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:42.317210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 169/347] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:42.318107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.384779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 179/348] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:42 2022] POST /identity/v3/users => generated 246 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:42.384779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.384779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.384779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.399730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:42.401165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:42.434474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d9d29f-341c-41df-9936-b5a29b4110a0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.482419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 180/349] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:42.483454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.486898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.488468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.488752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.489027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.489699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:42.490009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.498772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.520054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 170/350] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:42.528168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.573115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.651468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf445b9-c5f8-42d6-a23c-db361703aaf7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.661297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:42.695189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.695586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d5ad84-1edb-4ca3-beda-4e29b1cbc3e8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.696324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.696804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.697460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.698455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:42.698946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.701301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.716878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 181/351] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:42.727611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.745627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.746331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.746715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.747059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.748046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.750412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1e458-ddba-4e1d-aedc-814e4144b21c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.835302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8895f0-34ef-4ad9-bb62-253a1516f161 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:42.851125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 171/352] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:42 2022] POST /identity/v3/users => generated 248 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:42.860392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e20b-e746-415d-a120-81da8d46b270 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.861245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e20b-e746-415d-a120-81da8d46b270 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.861529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef27e20b-e746-415d-a120-81da8d46b270 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.863102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 172/353] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:42.864821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27e20b-e746-415d-a120-81da8d46b270 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:42.878230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:42.878913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:42.879214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:42.879598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:42.880258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:42.881774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53446ff4-7d42-4c34-aae8-88b5f66c0780 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:42.981688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:43.003323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 182/354] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:42 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:43.005307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.005813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.006347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.024548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef27e20b-e746-415d-a120-81da8d46b270 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:43.043213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:43.047519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:43.069892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:43.074720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:43.075622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.075911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.076194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.088782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:43.090290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:43.107855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:43.108794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:43.109631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:43.109631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:43.109631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.109631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 173/355] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:43.116785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:43.136284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 183/356] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:43.234891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00034-9f73-4fd4-bb2c-a98101fe5164 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:43.264102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:43.273868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:43.274576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.274883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.275156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.275809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:43.276114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:43.277499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:43.290326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 174/357] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:43.295921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:43.400226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2d715a-5537-4c17-b0bb-9a1a2610f9cd None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:43.437917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:43.438584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.439202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.439624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.440334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:43.441769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:43.444677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:43.530395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 184/358] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:43 2022] POST /identity/v3/users => generated 249 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:43.573818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d174890-764e-41ea-8f46-92defe0b5429 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:43.613072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:43.614072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.614623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.615123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.616023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:43.617658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:43.664195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 14:45:43.664195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 21 14:45:43.664195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [parameters: {'id': '947b9084530b44bfaade12c45b82417e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 21 14:45:43.664195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:45:43.682578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return ref.to_dict() Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:45:43.683562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 14:45:43.684478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 14:45:43.685323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:45:43.686234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:45:43.689700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': '947b9084530b44bfaade12c45b82417e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:43.691917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 21 14:45:43.693830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:43.697817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 175/359] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Oct 21 14:45:43 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:43.704025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:43.732961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:43.874403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698c8d49-8ef1-4089-9333-fd0620308597 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:43.920343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:43.921005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:43.921311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:43.921584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:43.935019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-7f7c2e64-7f39-4255-8cc3-3488ea9371a6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:43.940936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:43.943013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:43.944947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 185/360] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:45:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:43.954236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:44.025728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:44.080664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b8f23-35c8-4fa8-a03d-3a9108dc2a77 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:44.123572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:44.124590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:44.125089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:44.125561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:44.126514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:44.127053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:44.128675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33879a-865c-443c-893e-3b9641899351 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:44.137443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 176/361] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:44.899642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468eb2-f175-4d5f-81c2-276ccb228b6f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:44.900194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468eb2-f175-4d5f-81c2-276ccb228b6f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:44.900194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a468eb2-f175-4d5f-81c2-276ccb228b6f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:44.901696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 186/362] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:44.907211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbf711-6072-4548-b1ca-73f7001db4c0 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:44.908221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbf711-6072-4548-b1ca-73f7001db4c0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:44.908303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbf711-6072-4548-b1ca-73f7001db4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:44.933828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-13cbf711-6072-4548-b1ca-73f7001db4c0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:44.935055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-13cbf711-6072-4548-b1ca-73f7001db4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:45.036505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 177/363] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:45.465312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a468eb2-f175-4d5f-81c2-276ccb228b6f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:45.476871 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4fdb-8d34-440e-994d-868881409357 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.477161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4fdb-8d34-440e-994d-868881409357 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.477426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8be4fdb-8d34-440e-994d-868881409357 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.479066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 178/364] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:45.483933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.484217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.484492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.503710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:45.504887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:45.562991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:45.576942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:45.593017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 179/365] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:45.630135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a468eb2-f175-4d5f-81c2-276ccb228b6f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:45.674507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:45.675507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.676020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.676504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.691557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-7d5aca4e-41ac-4200-8a7e-157405b8e106 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:45.697520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:45.698469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:45.699305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:45.706807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 187/366] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:45.708399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:45.849932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9346-4617-48c6-8ff9-21e640786670 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.850740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9346-4617-48c6-8ff9-21e640786670 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.851360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8a9346-4617-48c6-8ff9-21e640786670 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.854161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 188/367] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:45:45.858902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.859426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.859922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.861334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c105f-cbce-4d5e-baad-ec76eb4d5005 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:45.879155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:45.880804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:45.931388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:45.932064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:45.932295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:45.932570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:45.933244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:45.933550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:45.934965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:45.948970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 180/368] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:45.949898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:45.972726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 189/369] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:45.972726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.006875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.088760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.120992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331229b5-5ad3-49a1-9016-387c4b54cfc8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.167524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f115c4-2782-4ddc-b262-1195184a0126 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.193735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.194376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.194661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.194937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.218445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:46.227533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:46.228430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:46.229223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 181/370] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:46.234529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.258569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.259243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.259515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.259783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.280244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.282987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:46.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:46.315704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:46.317657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:46.317657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:46.317657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:46.317657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 190/371] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:46.317657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.374935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.383306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71996107-2cac-4a7f-87b1-224dc3a21b36 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.431130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.431982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.432257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.432589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.433856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:46.434281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.435989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.446994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 182/372] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:46.451294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.461229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae7d57e-5a8f-4224-9c75-b7dc213c041b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.475846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.505468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.506109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.506392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.506722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.519980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:46.526113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:46.527135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:46.528087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 191/373] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:46.532683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.574108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fb0cb9-95da-42c0-a197-0045e8446f34 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.610293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.611600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.611887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.612159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.612804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:46.613107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.614580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3647e2f4-c9f7-47d9-b89d-f8a774a11f8d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 183/374] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca77d54-1f76-49f4-9074-0e4fea334b16 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca77d54-1f76-49f4-9074-0e4fea334b16 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca77d54-1f76-49f4-9074-0e4fea334b16 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 184/375] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:46.632547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca77d54-1f76-49f4-9074-0e4fea334b16 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.645228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea78782-cd0c-4c7d-94e1-6492c1b4c8a7 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.652546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.682421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.683086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.683366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.683670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.695837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.697305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:46.713150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:46.714094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:46.714985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:46.714985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:46.714985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:46.714985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 192/376] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:46.715197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.740903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.745248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca77d54-1f76-49f4-9074-0e4fea334b16 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.784316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.784924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.785223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.785470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.786131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:46.786436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.787828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce4544-c37a-4c7c-b182-29b38cec407b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.796093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 185/377] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:46.801907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.802204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.802468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.821066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec615b8-532e-41ee-a59d-e1f2cef5421d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:46.821241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:46.822306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:46.858018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:46.858515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:46.858808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:46.859082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:46.867743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:46.869203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:46.882748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:46.883634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:46.884533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 193/378] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:46.885589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.905481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 186/379] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:46.906705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:46.932754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:46.994540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.015799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc3ab-a09d-4503-8149-a0d466ae5b2e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.058965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.059563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.059846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.060119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.060767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.061068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.062463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.070299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd33931-a62b-46ba-b0a6-e73b7b8ab67e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.081178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 194/380] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.081178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.120675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.121404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.121747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.122077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.135863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.137711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:47.153889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:47.155102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:47.155934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:47.155934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:47.155934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:47.156490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 187/381] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:47.158088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.197333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be883038-bb63-41f7-9760-4009efc5561b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.223075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.231724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.233406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.233406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.233406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.233723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.234030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.235510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a348c2-c09e-419e-a1b6-151e299fe6a5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.262001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f4bde8-0e71-495f-b082-a38954da7165 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.268952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.272913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 195/382] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.274105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746877-c38d-4f15-ac10-c98b1450a9a6 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.274389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746877-c38d-4f15-ac10-c98b1450a9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.274698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f746877-c38d-4f15-ac10-c98b1450a9a6 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.275895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 196/383] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:47.276592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746877-c38d-4f15-ac10-c98b1450a9a6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.303368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.304032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.304311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.304585 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.312011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.313453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:47.325866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:47.326822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 188/384] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:47.327750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.380583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f746877-c38d-4f15-ac10-c98b1450a9a6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.415091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.415622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.415905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.416178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.416821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.417126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.418526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.428393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e2c57-b87f-4be6-bb89-ce57f6bbf82f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.429807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 197/385] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.430439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.456486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.467715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.468341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.468617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.468892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.480649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5eb6a850-9656-48e7-8fc2-fbcaa88d749f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:47.485478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:47.486618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:47.487542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 189/386] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:47.488179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.488179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.488453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.492578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.507666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:47.508866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:47.535163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a33d6d-9ec5-4bfa-87a5-fd2df8849e06 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.566853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.567468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.567768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.568036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.574059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 190/387] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:47.574963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.578370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.580665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:47.597327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:47.598269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:47.599599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:47.599599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:47.599599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:47.599599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 198/388] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:47.600047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.690593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146583a-16bc-4cbc-9165-655ee40f43e6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.703933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.717324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276d15e9-567d-4f17-ad16-c619deea0d85 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.724669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.725282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.725555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.725819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.726546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.726879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.728322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.731440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:47.752710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.753340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.753621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.753894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.762498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.764769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.766450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 191/389] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.768061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:47.784800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:47.785902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:45:47.787022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:47.788838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 199/390] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:47.789758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.888463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f26e14-55fc-4a8c-86e0-75d7ca3a4794 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.903028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b8274a-451d-46c7-9cbe-a9ebe315ad78 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:47.925456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.926070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.926352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.926622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.927479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.927793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.929806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.940468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:47.940708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 192/391] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.941494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.942022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:47.942452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:47.942926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:47.943582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:47.943888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:47.945303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:47.962521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 200/392] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:47.962521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:47.987196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.003777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.059422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd498ca-845a-4e8b-a40d-f47ecb1b7f8d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.081772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7cc3-f4e7-4f66-8e67-c8fd2e4f5fc6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.100361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.100994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.101273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.101549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.111874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.113370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.118420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.119074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.119349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.119613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.120636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:48.120946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.122701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:48.128915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.129805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:48.130693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:48.130693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.130693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:48.130693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 193/393] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.130942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.143177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 201/394] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Oct 21 14:45:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:48.144233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.230831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.237319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.245876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd888a0-566e-4124-8709-b9d4e242d480 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.246654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfc4467-6074-4e52-a5b8-0c565d3b1088 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.279943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.280121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.280568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.280662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.280845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.280947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.281116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.281268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.291956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.292736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.293418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.294171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:48.309224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.310288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:48.311178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:48.311178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.311178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:48.311355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.312191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:48.313031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:48.313031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.313031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:48.313031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 194/396] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.313261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 202/396] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.313261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.313381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.410680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143caa69-5237-43d7-8f1f-30c75b5edf2a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.412219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38c90e-3e51-4c9e-a7a0-a7009f7329a9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.444252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.444903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.445183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.445457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.445937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.446538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.446850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.447124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.447765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:48.448291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.449429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.451682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.452209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.453627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.461984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 195/397] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:48.470835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:48.471001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.472468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 203/398] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.473329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.564131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50560044-2cb3-4ec4-ae97-153d0df029fa None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.570241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8856-7a36-4663-9044-dea345c06c84 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.597828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.598451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.598764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.599037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.603121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.603714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.603992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.604262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.604897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:48.605199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.606567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.608883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.610326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.618005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 204/399] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:48.618881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:48.625281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.626784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:48.627760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:48.627760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:48.627760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:48.627760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 196/400] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.627983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.680362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.680949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.717956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2520692d-2ecc-4c9c-a0ff-feee56a6afc4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.731542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85f522-636e-4cd7-81f5-a59deffbfa12 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.753092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.753686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.753968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.754237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.754891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:48.755192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.756540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.764995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.765613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.765894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.766163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.768962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 205/401] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:48.769760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.777736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.779243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:48.791923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:48.792792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 197/402] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:48.795364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:48.870990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b5883-bf8d-45c5-a1e1-4b52843d1821 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.892535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d51a8-a6b1-438c-bc94-a8f684f5e53e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:48.893396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.901442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:48.904290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.904887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.905165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.905437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.906113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:48.906415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.908428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.926795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:48.926988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 206/403] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:48.927176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:48.927457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:48.927727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:48.928396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:48.929842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:48.932161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.021872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 198/404] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:45:48 2022] POST /identity/v3/users => generated 247 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:45:49.023746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.028659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26d929-2f69-46bd-ac04-043f78b6f438 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.069491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.070441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.070749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.071166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.071822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:49.072124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.073520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.090457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 207/405] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:49.090457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.137350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.152310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9acef8f-10a6-4dec-8d15-de01274d2c32 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.193348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.194103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.194228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.194496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.202855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.206176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.206300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948ddac1-b80d-458f-9949-7e1c327cd8a4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:49.216619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:49.217578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:49.218615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 199/406] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:49.219295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.246802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.247424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.247694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.247965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.248651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:49.248958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.249443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.250911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.269507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 208/407] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:49.270819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.327865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c9ce3-c4bf-4b90-9867-58515f6c5f0b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.364152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.364967 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.365347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.365722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.381949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=9ee990610dd642059936212305d94fb6, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.383230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ba4b8-6168-4b2d-b414-bffe6e489b0e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.383707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.384919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.418548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.419129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.419409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.419680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.426564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.428023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:49.439697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:49.440706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 209/408] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:49.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.534664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 200/409] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:49 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:49.534664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.551485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf1a970-4003-41a4-bf33-3c5f731874aa None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.595788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.596593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.596975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.597298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.598131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:49.598528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.600123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.620471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 210/410] 10.222.0.60 () {56 vars in 1139 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:49.621125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.621747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.645890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.653315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b781410-19b6-47f5-b44c-2a894b5247fc None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.694815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.696053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.696634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.697141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/fccd7b631a914ab684da6d642501cb64/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.716808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=fccd7b631a914ab684da6d642501cb64, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.718718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.737681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c1389f-59a8-426a-9693-557a6725f8c3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.777080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.777694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.777972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.778257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.778974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:49.779278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.780716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:49.793269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 211/412] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:49.794522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.794789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 201/412] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:49 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/fccd7b631a914ab684da6d642501cb64/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:49.800387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:49.906396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:49.930139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b451cb-5588-486c-8736-9d379aa1dace None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:49.972616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:49.973622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:49.974184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:49.974748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:49.989642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:49.991482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:50.008223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:50.009243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:50.010098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:50.010098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:50.010098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:50.011942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 212/413] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:50.018208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:50.027679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3de4792-e8b9-4b04-a8be-58319d5843c0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:50.055848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:50.067093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:50.067945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:50.068280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:50.068612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/33932a27fdc0413fac63db9898ef8475/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:50.086316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=33932a27fdc0413fac63db9898ef8475, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:50.088222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.132468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28197a9e-b942-4565-b3f7-3b53d9499e72 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:50.161664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 202/414] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:49 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/33932a27fdc0413fac63db9898ef8475/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:45:50.164625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:50.169790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:50.170426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:50.170737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:50.171008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:50.171665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:50.172057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:50.174170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.187565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 213/415] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:50.193910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:50.308911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5ffe5-fda0-4178-aa63-4b27df7593d7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:50.346761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:50.347411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:50.347690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:50.347961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:50.355905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:50.357374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:50.369932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:50.370838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:50.371676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 203/416] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:50.378103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:50.489805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aaf512-a152-427a-bc1c-6bb394952156 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:50.526017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:50.526646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:50.526935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:50.527204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:50.527898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:50.528200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:50.529683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94c26c-6836-4163-ade3-9bfe925ae9a9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.542904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 214/417] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:50.551725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:50.588746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:50.663158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d21c7e-31b7-47f3-9635-7a942c5d1a2d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:50.707200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:50.707915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:50.708192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:50.708507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/ce92d16f3ee44e9eb2a65978edc3938e/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:50.732414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=ce92d16f3ee44e9eb2a65978edc3938e, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:50.733867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6473f5b-c4ae-43c3-b87f-0180a154893b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:50.823092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 204/418] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:50 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/ce92d16f3ee44e9eb2a65978edc3938e/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:51.398457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db0946-e09b-4742-937a-39c1ab268ad9 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:51.398968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db0946-e09b-4742-937a-39c1ab268ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:51.398968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21db0946-e09b-4742-937a-39c1ab268ad9 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:51.400520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 215/419] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:51.405867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:51.406271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:51.406435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:51.430895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:51.432146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:51.535354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 205/420] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:51.858689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db0946-e09b-4742-937a-39c1ab268ad9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:51.965297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:52.032113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21db0946-e09b-4742-937a-39c1ab268ad9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:52.068014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:52.068647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.068925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.069198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.079817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:52.085075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:52.085950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:52.086763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 216/421] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:52.093523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:52.119130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:52.221935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58ef75-527b-4199-8f97-bfe05845febe None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:52.257119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:52.257732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.258009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.258279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.258977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:52.259276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:52.260641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d244e6-4275-46f6-be2c-76315737955c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:52.268971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 206/422] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:52.274837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:52.387214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6de63f8-fb74-4ba3-8799-7fcfb08b695d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:52.408718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:52.424022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:52.424668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.429313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.429313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.437477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:52.439502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:52.455909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:52.456901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:52.457826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:52.457826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:52.457826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:52.457826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 217/423] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:52.463077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024c0e-6678-48e4-aba3-fd87e246460a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.463693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024c0e-6678-48e4-aba3-fd87e246460a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.463693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5024c0e-6678-48e4-aba3-fd87e246460a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.465355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 207/424] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:52.467366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:52.470394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e9b3-4730-4d96-998a-93c77089c66f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.470691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e9b3-4730-4d96-998a-93c77089c66f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.470968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e201e9b3-4730-4d96-998a-93c77089c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.492211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e201e9b3-4730-4d96-998a-93c77089c66f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:52.493613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e201e9b3-4730-4d96-998a-93c77089c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:52.601393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 208/425] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:45:52.602669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3da2-5802-4260-bb56-f405670c1674 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.607008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3da2-5802-4260-bb56-f405670c1674 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.607388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9b3da2-5802-4260-bb56-f405670c1674 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.609019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 209/426] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:52.623311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.623588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.623836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a66e8b0-540d-4efd-91d7-f47f2313f74d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:52.623960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.646844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:52.660308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:52.660987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.661267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.663658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.663658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:52.663658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:52.663867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:52.664020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:52.664822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:52.679951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 218/427] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:52.686222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:52.714280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:52.802357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 210/428] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:52.821134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5738bc39-2d6f-4633-bdaf-751992ca186f None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:52.867846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:52.868489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.868762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.869025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.879276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:52.880775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-84a85aee-a8a7-4f3b-9545-90a7fc9b6fe5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:52.900524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:52.901405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:52.902288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 219/429] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:52.907207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:52.987079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cd0f-25c6-4d92-a2b2-8d881e7c932f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:52.987631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cd0f-25c6-4d92-a2b2-8d881e7c932f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:52.988104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f851cd0f-25c6-4d92-a2b2-8d881e7c932f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:52.989795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 220/430] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:52.994358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851cd0f-25c6-4d92-a2b2-8d881e7c932f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.048306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2b53c-8ce4-488c-9340-745c2611e081 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.088796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.089779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.090158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.090543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.091429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:53.091885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.093399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeabc87-616a-402b-b780-2d1c2b08ffc4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.097272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.103208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.117516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 211/431] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:53.117516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.117516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.117516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.133616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f851cd0f-25c6-4d92-a2b2-8d881e7c932f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.147807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:53.149482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:53.171775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.172684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.173034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.173845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.185466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:53.192451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:53.193528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 221/432] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:53.199279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.234599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 212/433] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:53.235512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.319876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cc17f2-2db3-4ac6-a190-2132f09f46df None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.356142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.356899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.357202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.357541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.359737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3877aa2c-4ab4-47b7-a0f5-309f6ec9a5df None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.369553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.371179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.374159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:53.387590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:53.392838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:53.393713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:53.393713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:53.393713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:53.393713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 222/434] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:53.393713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.397227 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.417738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.419335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.419788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.421582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.436303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:53.443046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:53.444185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.445158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:53.451061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 213/435] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:53.451980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.529292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15599d2-9f3a-4819-b66d-7ac6f27e5062 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.556315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12123f-68c0-4f7d-a1bb-ae1fd848ad34 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.565078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.565683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.565968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.566274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.566932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:53.567239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.568624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.581935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 223/436] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:53.581935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.597698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.598417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.598724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.598996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.599751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:53.600078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.601529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.614149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 214/437] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:53.615764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.644930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.663202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.694444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e4c4d-6e9b-498a-9ac3-818e4b4ce4fa None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.723209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bc796-72a5-4582-b669-ec7625c1fa4c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.732546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.733153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.733434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.733735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.734386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:53.734715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.736123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.744601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 224/438] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:53.745235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.764325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.765275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.765552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.765873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.778486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:53.783809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:53.784741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:53.785570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 215/439] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:53.786893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.856329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9525b-46ff-4d1b-8e93-e3d60b0ae26b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.927386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.939768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230abd2-8dc1-4b2f-aa2d-1205b78983a6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:53.941122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.941756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.942033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.942311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.942481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:53.952882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.954975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:53.969293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:53.973239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:53.974088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:53.974088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:53.974088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:53.974088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 225/440] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:53.974088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:53.982728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:53.983363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:53.983942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:53.983942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:53.990733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:53.992169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b220500f-286b-48aa-972d-ecb4e07de1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:54.004335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:54.005172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:54.006044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 216/441] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:54.006862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a81c-2773-449c-9cc3-eb2f7351dba1 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.007141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a81c-2773-449c-9cc3-eb2f7351dba1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.007414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a81c-2773-449c-9cc3-eb2f7351dba1 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.008631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 217/442] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:54.009327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a81c-2773-449c-9cc3-eb2f7351dba1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.078707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becf922c-0244-4dcd-9c32-a1c14a3b4085 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.111295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a81c-2773-449c-9cc3-eb2f7351dba1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.111462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.112095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.112382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.112654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.123288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.124742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:54.139491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:54.140392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:54.141301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:54.141301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:54.141301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:54.141301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 226/443] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:54.141540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.149010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.149618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.149959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.150218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.150899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.151196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.152553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.160661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 218/444] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:54.161371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.166320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.189435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.243904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5084c1-52db-42f1-b88b-e41507600727 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.259823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af10ec-4755-490c-b19f-ec832e9c3bc7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.277187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.277785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.278059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.278326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.278990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.279289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.280658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab31b8f-5787-4b85-95f9-d300ade796f6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.292209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 227/445] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:54.293488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.293799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.294068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.297278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.298367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.299095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.299095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.299754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.300054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.301496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.312450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:54.314222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:54.319522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 219/446] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:54.320244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.382275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 228/447] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:54.383145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.398299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.402809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.418407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d802e-6066-4ee8-bb0d-1ee7c925d2f0 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.453226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.453830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.454147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.454487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.455162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.455465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.456827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.468118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 220/448] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:54.468733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.494028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d13b-a93c-4b0f-826f-260e404b0fbe None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.532021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.532730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.533208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.533208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.543681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.545117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:54.562993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:54.564400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:54.565232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:54.565232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:54.565232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:54.569281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 229/449] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:54.570101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.579961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f67bdd-6503-4e72-8dfc-3224fa5d6952 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.615748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.616465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.616638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.616911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.624508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.626534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:54.639702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:54.640597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:54.641478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 221/450] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:54.642736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.643954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.653090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.680926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999e642-ab2b-4593-a979-ce8b10a0fce3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.715533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.716187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.716467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.716741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/d34f518f5eef401da71b7624c9917d44/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.731402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=d34f518f5eef401da71b7624c9917d44, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.732844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.743135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4b63a3-9d45-4cbe-99f3-3145815c386a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.776480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.777036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.777313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.777579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.785141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.786581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:54.797522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:54.799250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 222/451] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:54.800192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.813372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 230/452] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:45:54 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/d34f518f5eef401da71b7624c9917d44/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:45:54.814936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.882324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.921447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb7481-6dfd-4a37-ad98-2072f46acd74 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.935169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c8d9be-0cf6-482d-b3b5-8676c04644cf None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:54.959622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:54.966477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.967354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.967984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.968271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.969208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.969514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.971959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:54.981571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:54.982238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:54.982455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:54.982754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:54.984359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:54.984677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:54.987363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 223/453] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:54.987363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:54.987541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.008625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 231/454] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:55.009472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.094783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f28114-2ae0-4653-b238-ba582c6c13e2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.116255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31397a8c-6a1d-4e25-b29d-93b271de2899 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.122950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.132791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.133486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.133759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.134026 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.144971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:55.150268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:55.151409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:55.152194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 224/455] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:55.153295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.157808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.158440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.158749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.159026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.159761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:55.160065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.161492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.179733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 232/456] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:55.180565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.213904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.268313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d944672-600e-4748-a02c-c2afcf9669f7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.285577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd021-7583-428f-b6b9-dfa669d241b8 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.303928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.304563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.304838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.305110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.312357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.314537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:55.326084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:55.327004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:55.327883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 225/457] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:55.328442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.330448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.331094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.331375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.331648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.379565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.406033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.407755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.435828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec901cc8-7e6f-431f-a4fc-c640d9a35117 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.440867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.478477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.479109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.479467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.479742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.480306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:55.480605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.482363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.490424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 226/458] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:55.495523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.519686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 233/459] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:55.519686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.606046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f6438a-353f-4f0b-9222-f1e218b8f9b8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.617288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.621521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edca5c3-85be-4a2f-b8e0-b26c7a5fe49d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.644168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.644706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.644986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.645253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.655489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.660929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.661736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.662184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.663489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.663489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:55.663650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.665044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.683951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 234/460] 10.222.0.60 () {56 vars in 1137 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:55.684256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.723802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.725269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.789474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c40d8-20c4-4b5a-b16c-ae79d5fc5047 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.825901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.826536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.826847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.827133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.834256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 227/461] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:55.835367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.843274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.844768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:55.847507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:55.862439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:55.863552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:55.864586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:55.864586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:55.864586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:55.868235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 235/462] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:55.869098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:55.915988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:55.943122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27853b-3d3f-40bb-aafe-1d932afea8cb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.979083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:55.979723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:55.980005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:55.980278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:55.981419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d1899b-8ed5-42d4-ba53-0de970a9532d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:55.991677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:55.993129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:56.008440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.009348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:56.010208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:56.010208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.010208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:56.010208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 228/463] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.010431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.022417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.023319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.023581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.023854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.073791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.099271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.100745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.116936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2072861-2b72-4471-ae8a-22efb707ae09 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.147949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.155390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.156074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.156367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.156621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.168112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.169543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:56.184508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.185422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:56.186308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:56.186308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.186308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:56.190949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 229/464] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.192083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.212984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 236/465] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:56.218072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.296126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9349266-1c2e-475f-abbe-f6b43ef6b8b5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.309633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.326207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27ad8b-844b-47f2-9cf2-0b93799600f0 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.330593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.331223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.331502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.331774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.332407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:56.332711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.334215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.350115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 230/466] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:56.350115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.366227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.366875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.367157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.367435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.368083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:56.368439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.369807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.383393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 237/467] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:56.384615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.387499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.460600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615b8bd-b833-4674-836a-092ea9d9f046 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.489896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a71e5e-2d06-41fb-90c8-987699b6ae7e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.500481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.501775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.502102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.502395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.503069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:56.503371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.504860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.520929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 231/468] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:56.520929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.530549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.531217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.531507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.531762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.543190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.544721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.555334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:56.560403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.561280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:56.562155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:56.562155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:56.562155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:56.566615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 238/469] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.567525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.624051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.633180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90c1b-4b8c-48ac-af0d-d4074d11b8a7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.668836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.669462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.669739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.670003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.672402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c823f0-05e3-4e38-91b2-7c20cc301829 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.678515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.679974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:56.691551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.692444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:56.693321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 232/470] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.694492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.715399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.715968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.716249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.716524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.723907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.725434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:56.737555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.738582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 239/471] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.739782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.791070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.802274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef6f18-ed29-42db-b9e8-957292664cf7 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.838098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.838760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.839036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.839305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.843479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850d7b3-67f0-4103-8ea1-86bb9c8dc16e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:56.847138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.848592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.859227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:56.859798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:56.862935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 233/472] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:56.863849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.890907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:56.891596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:56.891924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:56.892205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:56.892890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:56.893195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:56.894680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:56.914402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 240/473] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:56.914402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:56.981892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8de170-df6d-4ccc-8acc-5bcce5ab9c71 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.016508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.018081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.018357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.018645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.019370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.019675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.021095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.022282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5da68f-8a93-4155-9684-c69b831b8024 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.039169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 234/474] 10.222.0.60 () {56 vars in 1139 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.040050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.052475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.058613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.059243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.059520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.059789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.060510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.060809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.062220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.074182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 241/475] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.074993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.113202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.142477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d417d2-eae3-4ef0-a7f6-5f782d984880 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.176965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.177562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.177849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.178120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.178830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.179140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.179455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee450197-a505-41a0-ba4d-c8d5930d4588 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.180555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219d507f-ab48-4105-971d-e0b11031e49f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.192714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 235/476] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.193873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e501-db52-49d4-aa8b-f1a346e16058 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.194158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e501-db52-49d4-aa8b-f1a346e16058 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.194430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f9e501-db52-49d4-aa8b-f1a346e16058 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.195701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 236/477] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:45:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:45:57.196404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9e501-db52-49d4-aa8b-f1a346e16058 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.217403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.218031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.218311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.218580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.225226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.226682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:57.238696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:57.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:57.241990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 242/478] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:57.243806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.269418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.301510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f9e501-db52-49d4-aa8b-f1a346e16058 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.334290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.334913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.335185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.335451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.336042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.343898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a9693-a1a6-423a-bcd9-2079fb310645 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.380354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.380945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.381221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.381492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.382212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=9ee990610dd642059936212305d94fb6) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.382526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.383979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.390226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 243/479] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/role_assignments?user.id=9ee990610dd642059936212305d94fb6&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.391027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.408277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.409772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ae32a-b2c1-4806-a455-5674d01fe359 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.490205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.513415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5396350a-72d6-4574-8af8-ba76051c8bd4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.529075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 237/480] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:57.530542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.530958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.531302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.553241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:45:57.554469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:45:57.556598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.557211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.557488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.557755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.558478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=fccd7b631a914ab684da6d642501cb64) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.558843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.560280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.570861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 244/481] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/role_assignments?user.id=fccd7b631a914ab684da6d642501cb64&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.570861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.599136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.638656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 238/482] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:45:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:45:57.639537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.716810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3dc54e-5800-48c6-83e9-36bb3b881a3c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.757969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.758509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.758869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.759097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.759795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.760101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.761525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.776658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 245/483] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.782921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.790780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3f16bb-1b2b-4672-9009-2ff30367a8f4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.792419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.841265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e512e-819d-4108-80b0-153538e193ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.841899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e512e-819d-4108-80b0-153538e193ae None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.842176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e512e-819d-4108-80b0-153538e193ae None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.842448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e512e-819d-4108-80b0-153538e193ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.854901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e512e-819d-4108-80b0-153538e193ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.856550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e512e-819d-4108-80b0-153538e193ae None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-992e512e-819d-4108-80b0-153538e193ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:57.874065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:57.875151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:57.876435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:57.876435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:57.876435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:57.877125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 239/484] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:57.881911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e512e-819d-4108-80b0-153538e193ae None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:57.904973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:57.931949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5627a248-c3c6-444f-9792-bec8ea6ca3fc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:57.981421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:57.981421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:57.981421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:57.981699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:57.982862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=33932a27fdc0413fac63db9898ef8475) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:57.982862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:57.985182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:57.998791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 246/485] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/role_assignments?user.id=33932a27fdc0413fac63db9898ef8475&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:57.999807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.012907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e512e-819d-4108-80b0-153538e193ae None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.057079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.057709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.057987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.058257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.058953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:58.059628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.060805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.079611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 240/486] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:58.081120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.105643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:58.231441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccef9cc8-a3e7-4d44-a33a-00bb847c87c6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.269342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7257a98-2f91-4122-99c9-c2dd1c2b474f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.273212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.273849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.274126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.274391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.286682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:45:58.293369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:58.294303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:58.295150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 247/487] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:45:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:58.302642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.312125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:58.312282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.312830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.313107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.313373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.322670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.324111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:45:58.336909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:58.339054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:45:58.341092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 241/488] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:58.346691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.415109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb027-4014-41a0-96af-5e4de9e15463 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.422537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:58.462429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.463418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.463920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.464383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.465313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:58.465812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.467503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.477182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 248/489] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:58.483772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.509629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37319b3-90cb-4733-8493-acc8a4a7e213 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.580180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.581144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.581725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.582224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.584306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:58.584826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.587197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.607000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:58.612580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 242/490] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:45:58.619186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:58.675011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174dafd2-10e8-4833-853a-912630b7856c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.766467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1e71b-6d79-4b23-9385-3ed10bdd1981 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:58.783669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.784477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.784854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.785227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.801061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:58.806658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:58.807297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:58.807569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:58.807845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:58.808578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=ce92d16f3ee44e9eb2a65978edc3938e) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:58.808888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.811193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.817829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:58.817829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:58.818472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 243/491] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/role_assignments?user.id=ce92d16f3ee44e9eb2a65978edc3938e&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:45:58.846840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:58.848316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:45:58.849377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:45:58.849377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:45:58.849377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:58.849377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 249/492] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:45:58.853003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:59.030177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26886b29-2d0a-4b0b-ae30-3d275cd56660 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:59.071466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:59.072074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:59.072350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:59.072619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:59.073258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:59.073567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:59.075125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:59.089527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 244/493] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:59.096248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:59.208894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bf27a-d641-426a-9be4-cba9c00c21f9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:59.241681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:59.261788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:59.262430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:59.262733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:59.263010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:59.273437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:59.274969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:45:59.293238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:45:59.294362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:45:59.295311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 250/494] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:45:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:45:59.300458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:59.396764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:59.462203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d44de4-18bd-4c98-8a12-3a378082c87f None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:59.517203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:59.518395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:59.518701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:59.518975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:59.519706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:45:59.520008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:59.521430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:59.544449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 245/495] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:45:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:45:59.550407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:45:59.663083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2ea6-71de-4669-8957-6a12b5bb53c5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:45:59.704908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:45:59.705616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:45:59.705903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:45:59.706178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:45:59.773163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:45:59.802586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:45:59.804980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeba43-763f-42fe-b9de-3337906ef0e7 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:45:59.961789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 251/496] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:45:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 21 14:45:59.967161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.118134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb65825-51fb-47d9-9b3b-7e07ab4086f6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.139125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38486f-408d-4d59-b797-8d8f6b5ed268 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.139457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38486f-408d-4d59-b797-8d8f6b5ed268 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.139672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38486f-408d-4d59-b797-8d8f6b5ed268 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.141129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 252/497] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:00.144363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:00.152094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4adc57-bccb-46ee-ba7b-02bba19ab915 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.152377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4adc57-bccb-46ee-ba7b-02bba19ab915 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.152658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4adc57-bccb-46ee-ba7b-02bba19ab915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.187420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7d4adc57-bccb-46ee-ba7b-02bba19ab915 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:00.189088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7d4adc57-bccb-46ee-ba7b-02bba19ab915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:00.196429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.197073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.197347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.197614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.218150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:00.220145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:00.242989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:00.243940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:00.244797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:00.244797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:00.244797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:00.253519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 246/498] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:00.259941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.325699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 253/499] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:46:00.420517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb8f8-0c63-45c7-8a18-6a9aeb47c113 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.455874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c340-7b4f-4fc2-b440-26453ec21eb2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.456177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c340-7b4f-4fc2-b440-26453ec21eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.456426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0c340-7b4f-4fc2-b440-26453ec21eb2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.458075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 254/500] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:00.462963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734565e7-f70b-4014-828f-655f92629f9c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.463246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734565e7-f70b-4014-828f-655f92629f9c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.463517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734565e7-f70b-4014-828f-655f92629f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.463863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.464471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.464745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.465013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.465649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:00.465950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:00.467317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:00.480569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 247/501] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:00.482565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-734565e7-f70b-4014-828f-655f92629f9c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:00.483811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-734565e7-f70b-4014-828f-655f92629f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:00.487263 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.494971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:00.539039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:00.624707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 255/502] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:46:00.634014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734565e7-f70b-4014-828f-655f92629f9c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.670916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38d185b-84a5-4f9c-b35c-7acbac5bf20a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.738753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.739777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.740256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.740828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.759505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:00.760668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734565e7-f70b-4014-828f-655f92629f9c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.761485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:00.780903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:00.782310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:00.783236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:00.790883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 248/503] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:00.796234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.796802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.796932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.797134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.797405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.805872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:00.808862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-200921da-1807-4c1f-85fc-1d265b80eed5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:00.818691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:00.819629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:00.820581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 256/504] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:00.821885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09700ed1-0103-4798-8a0c-358187df3dae None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.822172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09700ed1-0103-4798-8a0c-358187df3dae None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.822446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09700ed1-0103-4798-8a0c-358187df3dae None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.823984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 257/505] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:00.824697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09700ed1-0103-4798-8a0c-358187df3dae None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:00.921358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:00.943012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09700ed1-0103-4798-8a0c-358187df3dae None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.944692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8610ae29-ea21-449d-bcbf-060078853441 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:00.993032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.994287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.994741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.995148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.995990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:00.996439 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:00.996549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:00.997265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:00.997653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:00.998020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:00.998679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61b32-09a2-468f-9626-a45352f150f2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:00.998910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:00.999312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.000758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.014316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 249/507] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:01.014943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 258/507] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:01.016081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.016641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.017041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.017282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.037378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:01.038607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:01.086919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.118981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 250/508] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:01.120160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.131255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbb15-61df-4ab9-9166-49f1d1dc43f0 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.172827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.174026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.174292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.174700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.186282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.187476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:01.202119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:01.203129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:01.204106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:01.204106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:01.204106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:01.208223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 259/509] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:01.209062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.227320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.256811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7d74a1-f2e7-4afa-af52-d315c6364fbd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.296802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.298490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.299000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.300010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.314084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.316400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.325850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b230bd-9200-4ebe-9c75-dde11128556c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:01.338577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:01.339558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:01.340441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:01.340441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:01.340441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:01.342501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.343942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 251/510] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:01.345535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.364574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.365332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.365719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.366114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.366969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=d34f518f5eef401da71b7624c9917d44) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:01.367486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.369048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.375519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 260/511] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/role_assignments?user.id=d34f518f5eef401da71b7624c9917d44&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:01.377202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.482285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef53f88-903e-4cd1-8738-eca1206bf257 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.515526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.525589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.526459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.526908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.527350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.527533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0e3ed-b80a-47a2-b640-c458be26e805 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.528315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:01.528771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.530415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.543980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 252/512] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:01.545179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.571224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.571923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.572271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.572465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.573109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:01.573413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.574844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d8cb7-f923-4493-ae2d-6f17c5e5c29b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.587234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 261/513] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:01.588554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f04880-67c6-4f24-8b3a-ac000620b243 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.588846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f04880-67c6-4f24-8b3a-ac000620b243 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.589120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f04880-67c6-4f24-8b3a-ac000620b243 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.590459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 262/514] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:01.591058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f04880-67c6-4f24-8b3a-ac000620b243 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.638845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.654565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b65393-7dd9-4b10-998a-d5789f048c20 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.692034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.692677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.692960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.693238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.693850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.695267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.703602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f04880-67c6-4f24-8b3a-ac000620b243 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:01.707167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:01.708183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:01.709554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 253/515] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:01.710351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.747145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.748127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.748613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.749117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] PATH_INFO: `/v3/users/nova` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.757553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.759303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.763375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-1de5d087-c868-4c96-91ff-98edbb4e6eee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:01.774567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:01.775633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:01.776579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 263/516] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:01.777602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.777602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.777917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.802787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:01.803990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:01.817991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8538a-b951-4018-a5a6-39451398fc8a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:01.854811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:01.855412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:01.855683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:01.855952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:01.856655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:01.858129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:01.874942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 264/517] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:01.876483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:01.905830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:01.952609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 254/518] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:46:01 2022] POST /identity/v3/users => generated 254 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:01.953883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.038753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b8e46-3080-42fc-b51c-53e99d45a5d1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.113790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.114753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.115033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.115297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.115997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:02.116300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.117686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.132288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 265/519] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:02.138005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.172255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365d3395-9133-4588-8632-9462efaacc3e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.185935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:02.221951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.222799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.223154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.223426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.224154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:02.224457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.226128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.251799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 255/520] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:46:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:02.252203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.265898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:02.280006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee43673-1d76-4a99-8332-c321104da046 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.328314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.330315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.330899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.331313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.332246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:02.333018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.335005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.348762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 266/521] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:02.348762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.441291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562dc7f-0b0f-44f6-9d5c-6cef3d2c1b34 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.478665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43832189-60b8-4b40-b402-0b050c824bc6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.503974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.505001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.505485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.505954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.506752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.508322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.508535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.511362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.511362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.511362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:02.530669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:02.531706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:02.532609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:02.533346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.533346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.535545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 256/522] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/services/network => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:02.544460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:02.544641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:02.545526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:02.546408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:02.546408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:02.546408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:02.546408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 267/523] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:02.551953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.566780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:02.602617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:02.665912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f723-7177-434a-bad8-7f1e05a8d3a5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.711523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.712139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.712422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.712693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.713340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:02.713644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.717808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.736542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 268/524] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:02.742478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.746025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc30ab3-0cde-4ba3-ad0c-453979fa90a6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.803225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.803843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.804285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.804567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.805200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] RBAC: Adding query filter params (name=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:02.805504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:02.806976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:02.812733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 257/525] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/services?name=network => generated 124 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:02.819842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:02.854426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:02.913699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf74e23-918c-44f1-a9ee-77bf333e03da None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:02.981096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:02.984098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:02.984382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:02.984659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:02.988077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:03.000963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221eedd8-9b71-4618-94be-b16e65970b4b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:03.003153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:03.004621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:03.035507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:03.036471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:03.037371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 269/526] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:03.046038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:03.064221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:03.067382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:03.067860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:03.068269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:03.069096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:03.069534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:03.072214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:03.089349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 258/527] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:02 2022] GET /identity/v3/services?type=network => generated 124 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:03.191239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1abd0-a955-4b56-bd2a-05d197ef1ed4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:03.227390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:03.237314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:03.237931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:03.238212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:03.238486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:03.239209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:03.239516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:03.241133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bdfcc-1462-4e95-9340-dd9cdff74b6a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:03.260008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 270/528] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:03.266352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:03.409879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3036212-7b42-423d-88d0-670d73e31fb6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:03.457139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:03.457731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:03.458009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:03.458282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:03.460267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=9ee990610dd642059936212305d94fb6, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:03.461709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97355a8-2b2b-4bf4-ac05-94accfa70cbd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:03.530728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 259/529] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:03 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 14:46:03.842807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ad7f-3e25-46f2-82e3-4a4a2299a73e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:03.843148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ad7f-3e25-46f2-82e3-4a4a2299a73e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:03.843322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6631ad7f-3e25-46f2-82e3-4a4a2299a73e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:03.845245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 271/530] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:03.850860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f501-7072-41f9-8c9d-0b8daf652949 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:03.851156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f501-7072-41f9-8c9d-0b8daf652949 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:03.851498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7253f501-7072-41f9-8c9d-0b8daf652949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:03.871435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7253f501-7072-41f9-8c9d-0b8daf652949 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:03.872703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7253f501-7072-41f9-8c9d-0b8daf652949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:03.887541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:03.952075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 260/531] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:04.406073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ad7f-3e25-46f2-82e3-4a4a2299a73e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:04.455495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a19d67-27b8-4e8d-873c-898a91990597 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:04.456044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a19d67-27b8-4e8d-873c-898a91990597 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:04.456136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a19d67-27b8-4e8d-873c-898a91990597 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:04.457696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 261/532] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:04.463089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:04.463369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:04.463638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:04.483962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:04.485544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:04.559661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 262/533] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 21 14:46:04.564414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6631ad7f-3e25-46f2-82e3-4a4a2299a73e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:04.632226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:04.632872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:04.633149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:04.633419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:04.671239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:04.676936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:04.677881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:04.678744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 272/534] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:04.687726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:04.804988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:04.829336 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669a5e1-e963-4e48-86a5-b830629f2900 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:04.931665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:04.932197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:04.932472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:04.932736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:04.933458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:04.933758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:04.935136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:04.945061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 263/535] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:04.951929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:04.952079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.013437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.072645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c128267-5dd4-4f88-a427-ee2b020e1b8a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.078276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07267136-6830-41ef-91f4-6d2bccfcdff8 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.109295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.109930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.110498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.110935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.117791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.118418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.118727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.119004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.121809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-04ca7931-636e-49b5-9d43-2783474e4918 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:05.127856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:05.128769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:46:05.129580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:05.130195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.133590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.133806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.133965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 264/536] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:46:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:05.135251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33162-b16e-44e5-97e4-9d670ca89cf5 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.135533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33162-b16e-44e5-97e4-9d670ca89cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.135811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a33162-b16e-44e5-97e4-9d670ca89cf5 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.137321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 265/537] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:05.142218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a33162-b16e-44e5-97e4-9d670ca89cf5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-95fac993-9dd1-4406-909a-2e9124f39ae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:05.149606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:05.151731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 273/538] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.153756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.175797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:05.177002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:05.252390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a33162-b16e-44e5-97e4-9d670ca89cf5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.252964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 274/539] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:05.253823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.285311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.326444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.327097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.327378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.327649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.328314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:05.328617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.330837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.345820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 266/540] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:05.351692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.363202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86483e9-0bda-47b9-aec4-09b4c91415f0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.402890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.403614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.403967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.404314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.408619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:05.408619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.408619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.419999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 275/541] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:05.431652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.475608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.557552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509664d1-df09-4164-8d8c-e454485e6f20 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.567793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc28b1-6bbb-421d-aa14-18d5fa6823df None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.570833 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.602380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.603423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.603897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.604360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.612990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.614733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:05.626584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:05.627540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:46:05.628448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:05.634938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 276/542] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:05.642378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.643293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.643620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.644072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.644539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.665127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.667689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:05.686598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:05.687575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:05.688975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:05.688975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:05.688975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:05.691147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 267/543] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:05.692475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.774686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7582c-1509-4b6d-8e5f-55fd468eaf59 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.807400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120bb1c-2d9d-4ed7-88a2-4ba66d3c1956 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.816771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.817655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.818072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.818481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.819518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:05.819979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.821735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.827033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.849361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 277/544] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:05.849361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:05.858254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:05.859638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:05.860982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:05.861462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:05.862346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:05.862780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:05.864543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:05.958597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f993a34-75b1-4b3a-b7c5-c365e34c4163 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:05.962756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 268/545] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:05 2022] POST /identity/v3/services => generated 251 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:05.962756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.021209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.021835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.022111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.022381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.023322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:06.023859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.025285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b41d9bd-e112-4868-b85d-962f671355d6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.042422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 278/546] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:06.045010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29839d-f745-4447-9535-afb91c39ab88 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.045312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29839d-f745-4447-9535-afb91c39ab88 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.045587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29839d-f745-4447-9535-afb91c39ab88 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.047497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 279/547] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:06.048547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29839d-f745-4447-9535-afb91c39ab88 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.084023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b2fa-4ac1-4d67-9912-422a85356761 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.122970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.123670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.123997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.124357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.135738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.137793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.139364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-e9fd5ada-4a34-40e8-9df7-9ee7049719ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:06.156648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:06.158036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:06.158984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:06.158984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:06.158984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:06.159675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 269/548] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:06.160495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.160774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.161070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.187119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:06.187999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:06.197910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29839d-f745-4447-9535-afb91c39ab88 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.215203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.240317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.240962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.241240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.241512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.249595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.251649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:06.265877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:06.267129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 280/549] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:06.268708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.288811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 270/550] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:06.294991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.393370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c903fe6-4811-4684-ac84-76ea88856e0e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.404040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0de55b-4a9b-44da-a92f-a1844dd47f42 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.424976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.440134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.440750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.441029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.441302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.442369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:06.442727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.444345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2d4e4-6c18-4d5e-82c2-2c2999882137 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.447289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.448147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.448629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.449106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.450028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:06.450534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.452215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec4b572-897c-4b5c-a979-279815f789d0 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.457975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 281/551] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:06.459290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a55d-8742-47da-8217-546b489f2c35 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.459573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a55d-8742-47da-8217-546b489f2c35 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.459845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913a55d-8742-47da-8217-546b489f2c35 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.462817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 282/552] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:06.463783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913a55d-8742-47da-8217-546b489f2c35 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.482710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 271/553] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:06.484642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.485159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.485811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.512256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:06.513439 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:06.526667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.594413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913a55d-8742-47da-8217-546b489f2c35 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.597198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 272/554] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:06 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 21 14:46:06.598047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.636726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.637692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.638191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.638661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.646060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.648090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:06.660521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:06.661432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:06.662341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:06.664544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 283/555] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:06.665553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.714714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.741490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6cde0-924f-4144-b65b-0c5a834080b9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.778076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc914f-bd66-4943-b541-0240d3a63ebd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:06.787432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.788385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.788840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.789388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.801199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.803080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.806444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:06.819016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:06.819660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:06.820017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:06.820292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:06.820959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:06.822469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:06.823541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:06.824454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:06.825372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:06.825372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:06.825372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:06.827590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 273/556] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:06.829001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.890958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 284/557] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:06 2022] POST /identity/v3/services => generated 240 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:06.892961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:06.955488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b507c94-6582-449d-aad5-a3d7b4de733c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.009881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.010800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.011255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.011673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.012518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:07.013066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.014705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.030033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 274/558] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:07.031328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.035727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05632690-27ee-4760-be07-041e0e93c79b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.039567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.093243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.094014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.094299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.095496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.096176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:07.096514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.097983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.099153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.111917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 285/559] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:07.112725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.159555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501518e-5a8e-4fc2-88e4-432051af93f1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.201963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.203632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.204169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.204893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.218794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:07.226605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:07.227744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:07.228852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:07.231770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 275/560] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:07.232717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.248507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3f6f56-14c1-48fe-8b79-4f5f3a915ddd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.284918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.285761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.286139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.286510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/d34f518f5eef401da71b7624c9917d44/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.288714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=d34f518f5eef401da71b7624c9917d44, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.290251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.358658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b9d38d-d551-49b0-90f4-8b2440e52460 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.359137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 286/561] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:07 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/d34f518f5eef401da71b7624c9917d44/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:07.361151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.374777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.402558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee169152-213c-4110-948d-cecded7da700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.403326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169152-213c-4110-948d-cecded7da700 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.403605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169152-213c-4110-948d-cecded7da700 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.403874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169152-213c-4110-948d-cecded7da700 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.414261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee169152-213c-4110-948d-cecded7da700 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.416012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee169152-213c-4110-948d-cecded7da700 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.419675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ee169152-213c-4110-948d-cecded7da700 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:07.431103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:07.432972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:07.434184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 276/562] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:07.439848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee169152-213c-4110-948d-cecded7da700 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.568996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee169152-213c-4110-948d-cecded7da700 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.611939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.612560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.612839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.613173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.613870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:07.614226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.615774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.622792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c672be-847a-415b-9774-67b867e13520 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.634679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 277/563] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:07.643132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.663626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.664291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.664670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.664846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.665572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:07.665867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.667255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.683491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 287/564] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:07.686278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.689346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.820268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:07.833409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701d7f2-1e5f-49de-8643-12d537335ec3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.840980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60208b28-3bc1-4c97-bc93-0eb1a1920251 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:07.871612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.872411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.872790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.873185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.885557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.887599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:07.903318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:07.904507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:07.905593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:07.905593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:07.905593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:07.907554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 288/565] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:07.913194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:07.933047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:07.934003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:07.934385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:07.934788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/c97099704f6b486092571686ff58b3e6/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:07.991507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=c97099704f6b486092571686ff58b3e6, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:07.992975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:08.039473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c60fbb-a8df-4b2b-a3ef-fc9355bc7f40 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:08.074718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:08.075446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.075829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.076199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.076945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:08.077351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:08.078941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:08.091879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 289/566] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:08.092350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:08.097796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 278/567] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:07 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/c97099704f6b486092571686ff58b3e6/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:08.098862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:08.216625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3933a4-9d02-4c47-b805-20950d43a094 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:08.254131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:08.255192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.255681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.256182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.264444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:08.266189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-33ee77a3-8cad-4658-af9d-7c1714fbe747 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:08.280709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:08.281711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:08.282749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:08.285303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 290/568] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:46:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:08.290960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:08.311770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95dd7b-df87-4b1d-8799-5efea81e832e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.312088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95dd7b-df87-4b1d-8799-5efea81e832e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.312328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95dd7b-df87-4b1d-8799-5efea81e832e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.314531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 291/569] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:08.320041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.320333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.320586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.349738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:08.350907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:08.435915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:08.439410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:08.457438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 292/570] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:08.457681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc55f2b9-b7b4-4bbe-8f54-7beb2a9e6da1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:08.506199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:08.506834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.507110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.507376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.508184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:08.508487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:08.509899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:08.532966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 279/571] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:46:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:08.538267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:08.662741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b7872-825e-4713-a94c-b0aa7ac7e0dc None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:08.698756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:08.699419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:08.699696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:08.699962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:08.763076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:08.772081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:08.773781 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:08.838095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:08.887959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 293/572] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:08 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:46:08.893735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.005722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75b18c-585a-4bc4-b119-f12c14b57870 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.034983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.054064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8becf-b3ee-4a6c-bb0c-748fe9238a27 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.076808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.077504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.077786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.078057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.078646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.081338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:09.091582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:09.092658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:09.094074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 280/573] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Oct 21 14:46:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:09.100891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.119004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.145679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.146366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.146655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.146930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.169124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.170647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:09.199140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:09.200151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:09.201053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:09.201053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:09.201053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:09.203321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 294/574] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:09.215579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.272095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a864945-b629-4026-b1a3-b2cd0e6f90a4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.311705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.313576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.318413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.320491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.321358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:09.321897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.323847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.334046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 281/575] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:09.340157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.427033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.465428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0b461-879b-445a-9029-56da9c01b1f9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.465677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3f98c-0820-4c6c-a025-68198b8a747e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.506506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.507790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.508540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.509063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.510623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:09.511176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.513798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.519004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.519798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.520238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.520642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.521406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:09.521845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.524306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8390-6812-4dd5-a25d-4140d994eb60 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.526013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.531636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 282/576] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:09.533736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9be2a-a6b6-4d3d-8b5e-cb9ea27cb964 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.534261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9be2a-a6b6-4d3d-8b5e-cb9ea27cb964 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.534780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9be2a-a6b6-4d3d-8b5e-cb9ea27cb964 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.536043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 295/577] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:09.536826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 283/578] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:09.537022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.537989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9be2a-a6b6-4d3d-8b5e-cb9ea27cb964 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.659039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9be2a-a6b6-4d3d-8b5e-cb9ea27cb964 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.667479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb312d6d-5fe0-4ff7-8353-2f217e343f3a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.694428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.700678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.701673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.702145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.702571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.710796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.710932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.711470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.711754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.712028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.712735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.712850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] RBAC: Adding query filter params (interface=public, service_id=93d2be845dcf4f458db2a88528566f2d, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:09.713041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.714459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1647092-fb15-413d-afd3-4901c56d018f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.724336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 296/579] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/endpoints?service_id=93d2be845dcf4f458db2a88528566f2d&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:09.725706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.725995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:09.726146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:09.727081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:09.728209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:09.728717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.728790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 284/580] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:09.730183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.750595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:09.751876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:09.822583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:09.833230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 297/581] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:09.837563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:09.902002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e6951-a58e-4de4-9903-b27ef76152ea None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:09.982111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:09.983100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:09.983594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:09.984096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:09.984926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:09.987208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:09.998780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 285/582] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/services => generated 772 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:10.042873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0e244-c132-42fa-8023-eb7b55bd8cde None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.106409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.106985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.107259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.107524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.108226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:10.108527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.109960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.134178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 298/583] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:10.139721 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.140274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.215299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:10.229417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:10.273072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51505ad7-553c-4769-9fe7-21446c716550 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.290597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ae7263-a450-4efe-83db-d7be8b508f98 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.312082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.312715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.312990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.313889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] PATH_INFO: `/v3/services/network` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.314684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.316047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-90c4090b-7918-4fb2-85e5-1dcb87d85c0e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:10.325007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:10.326042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:10.326966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 286/584] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:10.327743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2102a0-171b-4100-8e4e-203bfa9b667f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.327743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2102a0-171b-4100-8e4e-203bfa9b667f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.327743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2102a0-171b-4100-8e4e-203bfa9b667f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.329982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 287/585] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:10.334270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2102a0-171b-4100-8e4e-203bfa9b667f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.334988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.335618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.335900 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.336233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.336971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=9ee990610dd642059936212305d94fb6) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:10.337389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.338977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24b557a-c85c-4083-a872-983b434d17c6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.350758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 299/586] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/role_assignments?user.id=9ee990610dd642059936212305d94fb6&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:10.352296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.352592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.352871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.399915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:10.401065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:10.445812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2102a0-171b-4100-8e4e-203bfa9b667f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.467111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:10.483183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.483806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.484077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.484343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.484983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] RBAC: Adding query filter params (name=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:10.485377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.486882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.492637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 288/587] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:10.515709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.531138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 300/588] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:10.662829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28073f8c-a7d7-4a6a-9b28-91f0f0858b02 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.713346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.714378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.714891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.715367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.716209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] RBAC: Adding query filter params (type=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:10.716711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.718271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8a2bb-fc38-4c67-9b06-0ba9658fd896 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.727296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 289/589] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:10.732983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.788894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45269d5-fe54-485c-94e5-29a673a7f688 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.789534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45269d5-fe54-485c-94e5-29a673a7f688 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.790031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45269d5-fe54-485c-94e5-29a673a7f688 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.790144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:10.791986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 290/590] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:10.796874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.797438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.798437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.831646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:10.833201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:10.854676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40efb5ed-3b83-45cc-b2b8-fa847e563fa8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:10.895483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:10.896099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:10.896372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:10.896641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:10.897339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] RBAC: Adding query filter params (interface=public, service_id=3dfa98f52fba40a09219ffef63188bca, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:10.898000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:10.899395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:10.902918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:10.906342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 301/591] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/endpoints?service_id=3dfa98f52fba40a09219ffef63188bca&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:10.908565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:10.947080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 291/592] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:10.948031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.079631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61b554d-9b7e-479c-98da-a7a2fc922f7a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.103953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afd1d4-22eb-44b7-a263-e570748400c5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.137525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:11.138453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.139237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.139520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.139912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.140527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:11.146653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:11.148417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 292/593] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/services => generated 772 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:11.175107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.176017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.176442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.176857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.195354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:11.200387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:11.201341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:11.202175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 302/594] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:11.208352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.284454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:11.342501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.363414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec2aee-3edf-4151-946f-4aba3316e14b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.408173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.409390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.409873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.410504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.411874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:11.412402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:11.414259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:11.424427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 293/595] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:11.429783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.466404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef54971-facf-4a30-83ef-5fe97173613c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.504747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.505379 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.505652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.506113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.518086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:11.523615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:11.524537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:46:11.525355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:11.527931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 303/596] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:11.543141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.563521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3c6f7-e104-445a-96bd-28f306712a3d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.576108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:11.607042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:11.631196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.631864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.632147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.632422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.658838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:11.660700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:11.677931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:11.678950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:11.679875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:11.679875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:11.679875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:11.682482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 294/597] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:11.691486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.746374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbcfa29-8de5-4cf5-91b2-665fda7bbc71 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.807347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.808005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.808279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.808543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.809199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:11.809493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:11.810937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:11.830106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 304/598] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:11.837304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.855286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06720e7-7bc7-4fbb-a606-8630c457ead2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:11.898011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:11.899441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:11.899717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:11.899982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:11.902324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:11.902655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:11.905304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:11.929657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 295/599] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:11.935647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:11.968745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.008136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.021274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7434507-3257-4882-a988-88dc1c87b013 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.087715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.089532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.091032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.091316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.105545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed33fa95-d570-4936-81b9-41263a7dca38 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.109683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.111867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:12.129954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:12.130951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:12.131851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:12.131851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:12.131851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:12.131851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 305/600] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:12.139583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.174539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.175213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.175482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.175752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.185444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.191506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:12.220944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:12.221942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 21 14:46:12.222948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:12.226877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 296/601] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:12.231137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.307252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1e76d-968f-461f-b51a-64f649a66660 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.348182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.348794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.349068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.349334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.349973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:12.350268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.351651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.374329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.380187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 306/602] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:12.385610 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.398321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.433494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49bd00a-b005-4a05-b0ca-7802a8be28e2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.491145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.491762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.492036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.493295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.494024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:12.494327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.496018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.530049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 297/603] 10.222.0.60 () {56 vars in 1135 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:12.539755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.569502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04015c9-c5ab-4f22-8540-230fb8cd1458 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.611127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.612901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.614529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.614962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.635305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.636986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-f7db64e7-f770-4714-8dad-60ebfa0ad639 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:12.656693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:12.657731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:12.658664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 307/604] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:12.662722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b860-707b-4487-b97e-467cba44611d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.662722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b860-707b-4487-b97e-467cba44611d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.662722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f772b860-707b-4487-b97e-467cba44611d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.662722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 308/605] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:12.664314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b860-707b-4487-b97e-467cba44611d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.714759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92876b3f-d9f8-4cb1-9dc7-03ceeaf08a0d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.748069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.750652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.752049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.752488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.752863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.771020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:12.786916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f772b860-707b-4487-b97e-467cba44611d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:12.825476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.827067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.831370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:12.832038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.832335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.832616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.833358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:12.833670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:12.835183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b30a8a-cf70-4f0f-9665-58a0cb9c7cad None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:12.855676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 309/606] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:12.857128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:12.857515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:12.857888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:12.876394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:12.877868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:12.953292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 310/607] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:12.955194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:12.970323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 298/608] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:46:12.971343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.041893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.074230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.122448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f7d47-9aa0-4eb3-a36d-c65b2ec861b2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.148808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd5bedc-3964-43f9-b26d-a0ed31ba0826 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.161456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.162108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.162470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.162773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.176052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.177492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.194560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.195197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.195468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.195737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-1dfe8ed0-b7b1-4091-9033-d290dbe92872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:13.195856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:13.196772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:13.197641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:13.197641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:13.197641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:13.197641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 299/609] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:13.201592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501a762-3f9f-4862-90df-2d659c4b0991 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.201592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501a762-3f9f-4862-90df-2d659c4b0991 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.201592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501a762-3f9f-4862-90df-2d659c4b0991 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.206127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 300/610] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:13.207173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501a762-3f9f-4862-90df-2d659c4b0991 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.282566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.284743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e787bc-6eaf-404f-b3c9-4625a487bd37 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.323155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.344611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501a762-3f9f-4862-90df-2d659c4b0991 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.345170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.383231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.383762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.384038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.384309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.384948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:13.385251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.386614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.396788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 311/611] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:46:13.396788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.396788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.396788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.407502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 301/612] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:13.408695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.413313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:13.414543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:13.494928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 312/613] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:13.494928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.518553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9f97e8-0fa2-4dd3-babe-87902ccd80d1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.560520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.561178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.561456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.561729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.562593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.565155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:13.577748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:13.579041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:13.579901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 302/614] 10.222.0.60 () {56 vars in 1129 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:13.580711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.597088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.609226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.629774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c241-0cc9-4079-bb43-08de914344f7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.665221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.665784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.666061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.666333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.678953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.680421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:13.697518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:13.698667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:13.699566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:13.699566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:13.699566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:13.699566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 313/615] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:13.699566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.711134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342b2b7-c1b6-4325-a857-80bc34576969 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.749759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.751277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.751930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.752204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.760678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.762550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8b2f2050-39e6-41e5-a83e-896183a07d89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:13.775030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:13.775916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:13.776812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 303/616] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:13.777285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3768-9739-4af2-afb8-ae5065248c4f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.777968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3768-9739-4af2-afb8-ae5065248c4f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.778078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e3768-9739-4af2-afb8-ae5065248c4f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.782954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 304/617] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:13.782954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e3768-9739-4af2-afb8-ae5065248c4f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.815587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19fdc0-bc29-4134-abd4-a0411442e7b9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.849064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.849733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.850014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.853494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.853494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:13.853494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.853773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.853861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.854404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:13.857405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 314/618] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:13.858177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.886641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e3768-9739-4af2-afb8-ae5065248c4f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.920898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.921507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.921768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.922036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.923264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:13.923561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.924914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:13.936774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 305/619] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:13.937579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:13.962085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45182669-62b2-4358-8ddf-6e112a6d2426 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:13.995823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:13.996453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:13.996732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:13.997012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:13.997583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:13.998999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-b54e6d18-05b5-42f8-b4a5-4dc505ec814b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:14.010215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:14.011357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:14.012501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.014778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 315/620] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:14.016438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.016799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.017166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.035616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:14.036354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a1775e-5410-48ef-8488-3ca93c70f50b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.036844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:14.065416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.071048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.071649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.076707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.076707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.076707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.076707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.076707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.086094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 306/621] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.086921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.101044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.109322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 316/622] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:14.110735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.193118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36d8f2-9db7-48c2-aa0b-c7685fa3d5e3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.224479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac5794b-a6cb-46d2-8d2a-e136a6a0d46f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.227171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.227858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.228200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.228537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.229231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.229600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.231055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.236650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 307/623] 10.222.0.60 () {56 vars in 1124 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.237499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.265554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.266823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.267443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.267443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.274442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.275978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:14.288776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:14.289924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:14.291142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.291895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 317/624] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:14.292298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.298431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.343216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfafaf5e-55ea-44c4-a0a4-7592b176667a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.348385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.378211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.378846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.379128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.379388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.380008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.380303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.381650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.390612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 308/625] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.390612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.393514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3865472-153a-4088-aa73-4ef0cccb15a1 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.427759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.428354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.428629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.428897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.429611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=d34f518f5eef401da71b7624c9917d44) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.429909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.431365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.437138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 318/626] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/role_assignments?user.id=d34f518f5eef401da71b7624c9917d44&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.437904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.507038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f583c-2d93-4ce5-934d-044ea73096c9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.527174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.546518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.547164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.547482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.547751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.549045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.550505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.555421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130c468-7aa7-4cfd-b299-d63374c50a33 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.574738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.593127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.593751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.594026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.594295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.595580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.596041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.597718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.611067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 319/627] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.611806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.661474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 309/628] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:14.662348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.740500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee12742-d033-4c76-8623-33ea674ad42d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.792743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.793827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.794293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.794790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.795568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.798085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.808817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d9283-91f6-4bfc-8a25-cfd7a59376e4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:14.810133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:14.811204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 21 14:46:14.812060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:14.820768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 320/629] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:14.822600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.853791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:14.857708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:14.857708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:14.857708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:14.857708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:14.857708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:14.858648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:14.863721 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 310/630] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:14.864462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:14.874249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.912543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:14.981450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b428207-01cf-4541-876c-ae7cb4c947dc None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:15.009006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befefd8c-ace6-4b1c-920c-a6ac60a3a2d1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:15.036292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:15.036928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:15.037496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:15.037776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:15.038620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=c97099704f6b486092571686ff58b3e6) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:15.038980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:15.040568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:15.050068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 321/631] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/role_assignments?user.id=c97099704f6b486092571686ff58b3e6&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:15.051170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:15.051799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:15.054889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:15.055201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:15.055914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] RBAC: Adding query filter params (name=network) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:15.056225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:15.060365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:15.067998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 311/632] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:15.073643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:15.231340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475b90e-3927-4232-942d-39d80f07bd52 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:15.233600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:15.278088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:15.279026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:15.279305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:15.279573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:15.280214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] RBAC: Adding query filter params (type=network) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:15.280512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:15.281941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475a831-d0d2-413c-a0df-5a264052298a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:15.288537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 322/633] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:15.295391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:15.436259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcd8aee-79a5-4c63-b28a-d7b2d1e7535a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:15.438064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:15.481116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:15.481735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:15.482011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:15.482282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:15.483031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:15.484404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81a61e0-4685-4781-aa16-651286cbbc50 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:15.591732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 312/634] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:15 2022] POST /identity/v3/endpoints => generated 337 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:16.903410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad356-fd23-46f8-9555-d6252d341b22 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:16.903963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad356-fd23-46f8-9555-d6252d341b22 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:16.903963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dad356-fd23-46f8-9555-d6252d341b22 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:16.905512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 323/635] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:16.911513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb5523-9d4c-4a82-8123-2b287fd6731e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:16.911934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb5523-9d4c-4a82-8123-2b287fd6731e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:16.912107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb5523-9d4c-4a82-8123-2b287fd6731e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:16.931180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8cfb5523-9d4c-4a82-8123-2b287fd6731e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:16.932371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8cfb5523-9d4c-4a82-8123-2b287fd6731e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:17.015246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 313/636] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:17.176085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7dac-18e7-4d79-a926-6590fc3e937d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.176390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7dac-18e7-4d79-a926-6590fc3e937d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.176654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b7dac-18e7-4d79-a926-6590fc3e937d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.178245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 324/637] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:17.183522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.184039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.184510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.203738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:17.205514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:17.277615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 314/638] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:17.303498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b7dac-18e7-4d79-a926-6590fc3e937d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.416251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b7dac-18e7-4d79-a926-6590fc3e937d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:17.451497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:17.452518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.453003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.453468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.455472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:17.457024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-170f08dd-0b72-486b-b3ae-fd593bb0838a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:17.466364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:17.467365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:17.468197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.476330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 325/639] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:17.479919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.481515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c3d5-129f-4a32-a478-5ede57149148 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.482544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c3d5-129f-4a32-a478-5ede57149148 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.483067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5663c3d5-129f-4a32-a478-5ede57149148 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.484857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 326/640] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:17.490443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.490750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.491029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.500323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:17.510065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:17.512066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:17.538294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:17.583476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 327/641] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:17.587565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.618579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c949230-0eff-4d56-850b-29a9bf4a64b4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:17.660897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:17.661909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.662400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.663020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.663986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:17.664531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:17.666245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:17.678059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 315/642] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:17.679816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.711041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d912f99d-3cc2-439a-9e25-8087ee9bee84 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:17.748940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:17.749982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.750557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.751048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.751978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:17.753545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-2d47f919-fbd5-4190-98e7-7ba350efe434 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:17.763167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:17.764360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:17.765203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 328/643] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:17.766077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9812c31-42b6-4566-9c64-7a02c8ef6060 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.766077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9812c31-42b6-4566-9c64-7a02c8ef6060 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.766348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9812c31-42b6-4566-9c64-7a02c8ef6060 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.771298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 329/644] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:17.771298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9812c31-42b6-4566-9c64-7a02c8ef6060 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.773418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:17.796131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140f55f-89ad-44a3-b101-03c113ab0cb9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:17.797286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:17.833941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:17.834574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.834851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.835129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.835756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:17.836058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:17.837407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca255fb8-3427-49f7-950f-1aaa78ef7b02 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:17.841933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 316/645] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:17.843083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.843363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.843629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.864235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:17.865467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:17.893368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9812c31-42b6-4566-9c64-7a02c8ef6060 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:17.938755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:17.939591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:17.946848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:17.946934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 317/646] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:17.947340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:17.947340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:17.947340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:17.947579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:17.947657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:17.959655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 330/647] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:17.966086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:18.038100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:18.068717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70855d3d-d796-42e6-a1ae-b42ad0592209 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.080502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:18.094660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fe112f-374d-4c37-8442-0cae109bf155 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.117029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.117651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.117952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.118197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.118886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:18.120800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:18.136403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.137288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.137729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.138144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.138957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:18.139456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:18.140950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:18.146482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 331/648] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:18.189404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 318/649] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:17 2022] POST /identity/v3/services => generated 269 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:18.222154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:18.350139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed162105-a7b9-4a88-8055-545a2f88371d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.395919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.396571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.396851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.397119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.409698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:18.415378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:18.416345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:18.417185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:18.417829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 332/650] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:46:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:18.423946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:18.533042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c1b96-2516-4475-959b-47ce4d3464a6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.569045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.570017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.570530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.571141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.572010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:18.572518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:18.573149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:18.575676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:18.585823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 319/651] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Oct 21 14:46:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:18.592225 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:18.609683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:18.697437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d3965-bbd2-4d6c-a0cd-86affbcc2c82 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.733766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.734694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.735941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.736376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.747500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:18.749148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:18.763759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:18.764955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:18.766067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:18.766067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:18.766067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:18.768499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 333/652] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:18.774794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:18.885567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a08b9-a1ea-497e-946a-485294b6d57e None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:18.923089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:18.923703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:18.923984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:18.924257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:18.924904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:18.925207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:18.926656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:18.939572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 320/653] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:18.945184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.019036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:19.056528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e5176-7a32-4d8c-bc38-2e0004b50dbd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.092195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.092834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.093116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.093386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.102004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.103559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:19.116639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:19.117672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:19.118765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 334/654] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:46:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:19.124159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.184869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:19.236482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c541c0-c827-4692-a787-1d7257a3c8a2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.270623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.271253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.271531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.271803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.272526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:19.273082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.274589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.292743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 321/655] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Oct 21 14:46:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:19.298902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.401496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af532796-cf30-4301-a61e-f8bab5f6b96e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.427014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:19.439909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.439909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.439909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.439909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.449527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.451102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:19.466154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:19.468381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:19.471230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:19.471230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:19.471230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:19.471230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 335/656] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:19.473809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.580353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcfcf4-7ed0-4011-b396-7b36afdbc3bc None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.598975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:19.615939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.616579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.616923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.617200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.617843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:19.618146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.619536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.631758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 322/657] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:19.637381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.742534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef46522f-5086-45b8-8426-bf316e5ae2f4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.778319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.778988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.779324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.779599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.786699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.788199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:19.800263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:19.802254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:19.804394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 336/658] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:19.807436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:19.914077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43887a-29a5-4e4f-9515-c366e82bc6e5 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:19.949954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:19.950615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:19.950919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:19.951247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:19.951949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:19.952258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:19.953774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3c9a3-c808-4e35-b414-7a4e39af39b3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:19.971012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 323/659] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:19.973843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:20.011168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:20.086050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee356-2126-466a-a664-7e431152ba54 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:20.121543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:20.122362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.122733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.123038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/c97099704f6b486092571686ff58b3e6/roles/1535167c92354699abb73e06ca6802ab` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.125262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=c97099704f6b486092571686ff58b3e6, role_id=1535167c92354699abb73e06ca6802ab)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:20.127442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33930005-e81a-4b47-b0d0-a24669c97ce3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:20.129158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf48a80-beea-4429-a5e6-861a7bb5089a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.129665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf48a80-beea-4429-a5e6-861a7bb5089a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.129833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf48a80-beea-4429-a5e6-861a7bb5089a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.132911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 324/660] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:20.139265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5824-cafa-4e20-b4b0-6cb13802e66d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.140644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5824-cafa-4e20-b4b0-6cb13802e66d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.140644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e5824-cafa-4e20-b4b0-6cb13802e66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.159757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cf8e5824-cafa-4e20-b4b0-6cb13802e66d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:20.161035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cf8e5824-cafa-4e20-b4b0-6cb13802e66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:20.190241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:20.205438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 337/661] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:19 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/c97099704f6b486092571686ff58b3e6/roles/1535167c92354699abb73e06ca6802ab => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:20.245641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 325/662] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:20.477061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efca2-3dfe-476d-b785-d9c611d4d589 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.477359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efca2-3dfe-476d-b785-d9c611d4d589 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.477611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efca2-3dfe-476d-b785-d9c611d4d589 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.479220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 338/663] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:20.484325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54c883-386b-4e19-8cc9-7c99b9c32a36 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.484617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54c883-386b-4e19-8cc9-7c99b9c32a36 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.484973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54c883-386b-4e19-8cc9-7c99b9c32a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.503644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-dc54c883-386b-4e19-8cc9-7c99b9c32a36 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:20.504816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dc54c883-386b-4e19-8cc9-7c99b9c32a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:20.536422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02efca2-3dfe-476d-b785-d9c611d4d589 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:20.576061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 326/664] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:20.577313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcc04-8dc1-4af5-8e9d-0b730fa77dfc None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.577595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcc04-8dc1-4af5-8e9d-0b730fa77dfc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.577867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bbcc04-8dc1-4af5-8e9d-0b730fa77dfc None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.580368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 327/665] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:20.585529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.585809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.586083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.609697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:20.610792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:20.647511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:20.658708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02efca2-3dfe-476d-b785-d9c611d4d589 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:20.684504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 328/666] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:20 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:20.699559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:20.700174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:20.700454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:20.700729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:20.703708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:20.703708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:20.809275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 339/667] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:20 2022] POST /identity/v3/services => generated 241 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:46:20.857101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:20.886889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:20.979454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617a79f-2948-435b-b88d-c7906242bdb1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:20.984109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:21.023744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.024673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.024673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.024867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.025484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.027477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.102053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 329/668] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:20 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:21.108093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df32260-67a2-4680-975f-fe8806d4afcd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:21.162907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.163892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.164319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.164792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.165689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.167281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:21.180145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:21.181169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:21.182785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 340/669] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Oct 21 14:46:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:21.187488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:21.276968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:21.300977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d892bcc-ed38-4d34-a626-1257afd7d0b9 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:21.339752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.340405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.340887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.341327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.342372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:21.342699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.344083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.349036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 330/670] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:21 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:21.354850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:21.373075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:21.470286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7adc9e3-274d-440b-b654-9863fb0a6e80 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:21.506213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.506847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.507128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.507397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.508022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:21.508325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.509786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.515173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 341/671] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:21.524200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:21.632675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0bbff4-11d9-431e-b04f-8bc84943d4b2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:21.669150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.670090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.670547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.671020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.671914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] RBAC: Adding query filter params (interface=public, service_id=46feae5ef5ce48ec92f951477826480e, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:21.672952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.674508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d75bf5-866e-4047-a42c-ae8fc7856ec9 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.682545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 331/672] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:21 2022] GET /identity/v3/endpoints?service_id=46feae5ef5ce48ec92f951477826480e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:21.688487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:21.785302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:21.798285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56147896-b1ab-4cbb-bafb-47d443eb77b0 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:21.837757 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:21.838370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:21.838671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:21.838952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:21.839555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:21.840911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d193ca-ab23-4cc8-a86a-881311311dfb None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:21.848206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 342/673] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:21 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:46:22.453000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aea19-3595-44b7-a062-2405fb80bc32 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:22.453607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aea19-3595-44b7-a062-2405fb80bc32 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:22.453607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959aea19-3595-44b7-a062-2405fb80bc32 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:22.455284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 332/674] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:22.460562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:22.460967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:22.461107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:22.480354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:22.481551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:22.558590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 343/675] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:22.828024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aea19-3595-44b7-a062-2405fb80bc32 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:22.838607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:22.938890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959aea19-3595-44b7-a062-2405fb80bc32 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:22.984524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:22.985476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:22.985759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:22.986032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:22.997654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8a0f39fe-daa4-4e4a-a131-be6badde7835 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:23.003472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:23.004471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:23.005309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:23.009901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 333/676] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:46:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:23.011456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.088873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.121575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8042dc-dac9-49d0-a010-6fb45eadb9be None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.137506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df1911-56cf-45d0-9887-0a37a5e5d2c9 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.137815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df1911-56cf-45d0-9887-0a37a5e5d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.138062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df1911-56cf-45d0-9887-0a37a5e5d2c9 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.141469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 334/677] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:23.145687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.145967 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.146240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.158375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.159027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.159306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.159577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.160249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:23.160554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.161917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.166855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:23.169060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:23.170675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 344/678] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:23.177807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.262385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.276980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 335/679] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:23.303740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bca0b-a088-455a-92b3-1dfb917ee64c None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.334355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.339938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.340517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.340793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.341061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.352098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.353548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-7a776bee-1c45-46e4-92cf-86a17a41227b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:23.369103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:23.370160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:23.371104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:23.371104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:23.371104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:23.375297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 345/680] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:23.378541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.486776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4980b-42bd-49fa-adac-c83b9296fa1b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.491373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a97fa-a7f9-4b0c-90a4-71015ff27215 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.491946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a97fa-a7f9-4b0c-90a4-71015ff27215 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.491946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a97fa-a7f9-4b0c-90a4-71015ff27215 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.493511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 346/681] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:23.499810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.500087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.500479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.522159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:23.523766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:23.523958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.524544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.524814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.525078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.525720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:23.526015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.527392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.539538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 336/682] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:23.546221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.602725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 347/683] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:23.603513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.611258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.668557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c184e8-4937-459f-86ea-dd6fcb7841a6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.716618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.717256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.717537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.717813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.729343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.731068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.732381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c6388-5d0e-497c-ab7d-bd149838aff6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:23.744830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:23.745845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:23.746868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 337/684] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:23.754207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.755217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.778157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.778845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.779124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.779442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.780033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.781389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:23.791607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:23.792657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:23.793509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 348/685] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:23.800269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.877913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a4907f-ce0f-4a4f-9da9-2d97bc13950a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.896226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:23.915099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.915413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98503304-5917-450d-ba9f-7983b1f75ce5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:23.915707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.915984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.916252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.917671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:23.918002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.919756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.938800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 338/686] 10.222.0.60 () {56 vars in 1149 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:23.939541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:23.963887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:23.964516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:23.964793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:23.965067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:23.965707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:23.966020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:23.967535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:23.973960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 349/687] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:23.980119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.024933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.057794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57c1b5-09d3-4746-b2a2-fc69843e9b30 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.087998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756d626-ffe1-4e55-b7fc-ba6934abaec8 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.091151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.091771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.092045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.092313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.092882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.094257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:24.102834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:24.103994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:24.104845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 339/688] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:24.105625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.124204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.124806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.125084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.126031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.143819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.193279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.194750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.203514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9b552-709f-41ff-a572-ed6428bfc545 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.235828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.237527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.238132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.238408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.238701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.239329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.239629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.240975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6bf61-561d-4105-9283-6aac7eee485e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.245803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 340/689] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.253512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3b2f5-0cba-4327-b85e-76dce512eedd None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.253512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3b2f5-0cba-4327-b85e-76dce512eedd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.253512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3b2f5-0cba-4327-b85e-76dce512eedd None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.253512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 341/690] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:24.253512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3b2f5-0cba-4327-b85e-76dce512eedd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.295714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 350/691] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:23 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:46:24.296792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.355643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3b2f5-0cba-4327-b85e-76dce512eedd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.357138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.390201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.390833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.391215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.391483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.392301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.392823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.394541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cae0d3-b1ed-4979-a3f4-0c73f232460a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.399404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 342/692] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.400483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705e9d0-b528-4120-9769-c3650961f4d6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.400592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.400800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.401073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.419332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:24.420541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:24.438065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.438764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.439043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.439312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.440007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] RBAC: Adding query filter params (interface=public, service_id=af2efbc7849c4b258676bf7d922ce9b4, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.440313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.441726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.446780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 351/693] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/endpoints?service_id=af2efbc7849c4b258676bf7d922ce9b4&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.447603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.455955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.492410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 343/694] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:24.493253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.555680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40459fa-70b6-4f5a-8a08-a5863d8e5fec None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.588950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.591099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.591964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.592393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.592813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.595319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b6a589-70b5-47b9-99d7-068b25ef6bb4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.604771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.606124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:24.621502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:24.622469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:24.623461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:24.623461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:24.623461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:24.623461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 352/695] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:24.623461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.631442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.632425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.633034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.633784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.634443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.634779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.636145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.641348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 344/696] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.642365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.696488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.741503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff6623-935e-4429-8bfa-d10c428a58e4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.758752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63533e-1788-4dcd-a6c9-3bf9c2c83f8a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.781061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.782274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.783111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.783382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.783980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.785322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.791039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 353/697] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:46:24.791799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.793798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.794413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.794725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.795000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.795653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.795956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.797343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.813968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 345/698] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.813968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.841680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.921405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09a07-fe82-441d-bbbc-12190e76af8b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.933245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0cebd-0558-46ab-8eec-981a14dd738a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:24.947589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:24.961715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.962439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.962839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.963180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.963995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] RBAC: Adding query filter params (interface=public, service_id=433adf324335405bbbaedeccc3d734ea, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:24.964423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.966236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.973077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 354/699] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/endpoints?service_id=433adf324335405bbbaedeccc3d734ea&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:24.974053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:24.982221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:24.982908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:24.983954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:24.984236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:24.984859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:24.986895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:24.997828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:24.998805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:25.000415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 346/700] 10.222.0.60 () {56 vars in 1132 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:25.001476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.089246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:25.096890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633f2d3c-6e7b-45c4-ae5d-3ba7b586d585 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.105168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c130a0-a079-481e-8b57-8b92282aa044 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.134102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.134765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.135042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.135311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.140669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.141467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.141879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.142155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.142807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.143173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.144197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.145046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.150810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 347/701] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:46:25.151420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:25.159961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:25.160987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:25.161979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:25.169462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 355/702] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:25.171278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.227187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:25.299898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2c19f-9915-4148-a7f3-587a72ba2997 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.306846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf29fb4d-cb94-435c-aed2-e70f9f4f6359 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.342229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.342901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.343245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.343518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.344293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:25.344921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.346011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.347425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.348203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.348619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.349045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.349669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:25.349970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.351359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.356666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 348/703] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:25.359776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 356/704] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:25.363745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.367358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.381077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:25.487836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371dedfc-6193-4fd8-a8ec-1d0064b200aa None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.496503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:25.503862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b032696a-78f5-4554-a272-856b4ad9f162 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.526502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.527275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.527701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.528026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.529107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] RBAC: Adding query filter params (role.id=1535167c92354699abb73e06ca6802ab, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=c97099704f6b486092571686ff58b3e6) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:25.529429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.531489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.546992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 357/705] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:25 2022] GET /identity/v3/role_assignments?user.id=c97099704f6b486092571686ff58b3e6&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=1535167c92354699abb73e06ca6802ab => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:25.550783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.551662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.552080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.552524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.553315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:25.553759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.555363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4c78-f833-493f-9ad4-31d0c6a14d20 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.561826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 349/706] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:25.567977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:25.683775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e90fb50-57fd-4d0b-8a4c-3276d2e10db2 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:25.722012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:25.723041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:25.723464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:25.723895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:25.724741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:25.726308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd4d3a-c095-4972-bd00-b5ccbc7e060f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:25.811266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 358/707] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:25 2022] POST /identity/v3/endpoints => generated 347 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:27.050449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94422e-887b-4de3-96c7-3aa20111047e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.051023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94422e-887b-4de3-96c7-3aa20111047e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.051023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94422e-887b-4de3-96c7-3aa20111047e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.052660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 350/708] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:27.058127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e3ed-e37a-4f8c-af19-ccebfb2f7853 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.058554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e3ed-e37a-4f8c-af19-ccebfb2f7853 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.058714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a83e3ed-e37a-4f8c-af19-ccebfb2f7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.077857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6a83e3ed-e37a-4f8c-af19-ccebfb2f7853 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:27.079181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6a83e3ed-e37a-4f8c-af19-ccebfb2f7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:27.099666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:27.156754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 359/709] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:27.410213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86cc2-aec4-4b1f-ad70-840c01fa9f15 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.410734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86cc2-aec4-4b1f-ad70-840c01fa9f15 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.411146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd86cc2-aec4-4b1f-ad70-840c01fa9f15 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.412859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 351/710] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:27.417894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.418288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.418692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.432502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86cc2-aec4-4b1f-ad70-840c01fa9f15 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:27.437632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:27.439649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:27.521811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 360/711] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:27.522007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:27.556423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd86cc2-aec4-4b1f-ad70-840c01fa9f15 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:27.592016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:27.593004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.593499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.593959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.595764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:27.597345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-62088d54-e25b-4541-b0a5-51c3f2b43cd4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:27.606371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:27.607397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:27.608237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:27.612108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 352/712] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:27.618678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:27.709013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:27.727907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f99ea-b0fd-4da6-b84a-fa8a5b809536 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:27.763940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:27.764563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.764843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.765115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.765752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:27.766645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:27.768027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:27.772106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaf50-bfa9-4351-bf90-2e19707a8433 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.772390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaf50-bfa9-4351-bf90-2e19707a8433 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.772668 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6aaf50-bfa9-4351-bf90-2e19707a8433 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.774621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 353/714] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:27.778319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 361/714] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:27.779730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:27.779841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.780088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.780361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.802847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:27.804405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:27.877173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 354/715] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:27.878662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:27.888526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab27ee-d774-433e-bf11-4cf762915751 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:27.933392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:27.934414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:27.934922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:27.935396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:27.936224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:27.936756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:27.938921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:27.946573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 362/716] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:27.956403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:27.971945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:27.984159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.023973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36399df-d016-40ec-981b-087e5312c6d3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.081412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.082901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.083369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.083820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.084583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.086160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.088650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823127ed-301b-430c-86fb-f883276cf65e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:28.097444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:28.098501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 21 14:46:28.099369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:28.101650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 355/717] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:28.107092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.133305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.133963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.134243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.134513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.135203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.136560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.220146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1520788-2cc4-4376-8ea8-07ac6d9939c6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.237415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 363/718] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:28.238178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.254734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.260724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.261350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.261638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.261907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.262559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:28.262895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.264307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed93f0-d063-4a4a-a8cf-e309e9d78fb5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.275648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 356/719] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:28.275648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6310c3-16c9-492e-9d3a-c696df19757e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.275648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6310c3-16c9-492e-9d3a-c696df19757e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.275648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6310c3-16c9-492e-9d3a-c696df19757e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.275648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 357/720] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:28.279409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6310c3-16c9-492e-9d3a-c696df19757e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.338046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.377664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d353478-2969-4479-85e7-0b79bbe64644 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.396044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6310c3-16c9-492e-9d3a-c696df19757e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.418755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.419093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.419293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.419560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] PATH_INFO: `/v3/services/image` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.420162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.421524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-3f2415cf-0d02-4267-bd70-170fbae40049 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:28.432110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:28.433298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:28.434322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:28.436399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 364/721] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:28.437696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.440415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.441448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.441448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.441448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.442078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:28.442375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.443601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.444442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.445515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.450702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 358/722] 10.222.0.60 () {56 vars in 1114 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:28.451549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.463529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:28.464732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:28.514367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.537576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 365/723] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:28.538735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.568188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08c44f5-a7b4-472d-a64e-c0ebc9f9dbcb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.610720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.612302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.613271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.613895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.615002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.615912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] RBAC: Adding query filter params (name=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:28.616600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.618475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.624608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 359/724] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:28.630907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.661918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0d044-3954-4dfb-9a11-9619b789840d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.699113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.700139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.700750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.701231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.702202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.704663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.737999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a029bf-cd19-492c-9a1d-cfe8710f801c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:28.764707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:28.777080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:28.777776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:28.778060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:28.778331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:28.778987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] RBAC: Adding query filter params (type=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:28.779287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:28.780644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:28.791087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 366/725] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:28.794682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 360/726] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:28.838398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:28.986527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d89054-e468-4898-ae0a-99c86be7cb11 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:29.002584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:29.027365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:29.027905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:29.028180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:29.028447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] PATH_INFO: `/v3/services/compute` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:29.029033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:29.030388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:29.040490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:29.041507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:29.042514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 367/727] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:29.047700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:29.158404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038e918-2b67-4411-968d-b2b9ff9df490 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:29.197619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:29.198577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:29.199109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:29.199569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:29.200401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] RBAC: Adding query filter params (name=compute) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:29.200892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:29.202434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb4a9dd-0a99-4250-b09b-2ff8df8721af None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:29.208625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 361/728] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:29.217649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:29.319961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04c3ad-aece-4a99-80e3-5149772ea8ab None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:29.355572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:29.356182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:29.356461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:29.356735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:29.357364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:29.357666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:29.359058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab601ebe-9b61-4797-8a31-89bee3af19e6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:29.364032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 368/729] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:29 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:30.476849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec70-7ec9-44ce-b96e-b7d158c9dd40 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:30.477367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec70-7ec9-44ce-b96e-b7d158c9dd40 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:30.477367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcaec70-7ec9-44ce-b96e-b7d158c9dd40 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:30.478993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 362/730] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:30.484255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:30.484724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:30.485396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:30.503973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:30.505153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:30.540446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:30.573383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 369/731] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:30.846307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaec70-7ec9-44ce-b96e-b7d158c9dd40 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:30.904166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:30.963415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcaec70-7ec9-44ce-b96e-b7d158c9dd40 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.011140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.012253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.012751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.013224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.014002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.015589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-6c93dc09-03c8-4d01-b116-0601ee6a559e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:31.026779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:31.027763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:31.028590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:31.031326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 363/732] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:31.039747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.111090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75527c-ab5e-4e8f-b3f6-2a57bae4af0a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.111267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75527c-ab5e-4e8f-b3f6-2a57bae4af0a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.111549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75527c-ab5e-4e8f-b3f6-2a57bae4af0a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.113095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 364/733] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:31.119038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006110-8a63-4609-9775-8686c98bd65c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.119609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006110-8a63-4609-9775-8686c98bd65c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.120060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b006110-8a63-4609-9775-8686c98bd65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.139686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2b006110-8a63-4609-9775-8686c98bd65c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:31.140959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2b006110-8a63-4609-9775-8686c98bd65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:31.154140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8919af-fd13-47f1-a15d-8adb3dfe8302 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.195239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.196018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.196412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.196769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.197470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:31.197846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.199308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.205545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 370/734] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:31.211159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.223098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 365/735] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:31.224844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec2ce6-e9eb-4899-aafd-90cca0619796 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.225314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec2ce6-e9eb-4899-aafd-90cca0619796 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.225803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec2ce6-e9eb-4899-aafd-90cca0619796 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.227772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 366/736] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:31.240501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.241551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.242337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.251245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:31.267597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:31.268925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:31.317423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:31.337704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb87cf-21bf-444b-bdf7-130728dfd482 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.354656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 367/737] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:31.381209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.381806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.382086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.382416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.383080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:31.383384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.384817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.390192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 371/738] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:31.396952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.493017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.502563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8771e9-3020-42c4-91c6-12d41e783a8a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.545016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.545655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.545932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.546203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.547001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] RBAC: Adding query filter params (interface=admin, service_id=af2efbc7849c4b258676bf7d922ce9b4, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:31.547908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.549531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.555492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 368/739] 10.222.0.60 () {56 vars in 1249 bytes} [Fri Oct 21 14:46:31 2022] GET /identity/v3/endpoints?service_id=af2efbc7849c4b258676bf7d922ce9b4&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:31.561126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.614547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:31.617485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:31.623167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448cc81-3aad-47dc-8e8d-0df5ebf6dded None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.661076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.661619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.661901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.662198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.662883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.664250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4323b52-e342-4d2e-8e5d-8a0cb82daf62 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.680930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b588dd-a6a4-4a19-9ecb-93a06fe8cef4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.714950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.715547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.715823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.716136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.716728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.718076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792709f8-f593-469f-8b97-80254acd2bd7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.723984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 369/740] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:31 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 14:46:31.725193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a25e3-631a-44bc-9689-9a783b630e49 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.725479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a25e3-631a-44bc-9689-9a783b630e49 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.725758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403a25e3-631a-44bc-9689-9a783b630e49 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.731157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 370/741] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:31.731157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a25e3-631a-44bc-9689-9a783b630e49 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:31.745744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 372/742] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:31 2022] POST /identity/v3/services => generated 242 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:46:31.745744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.745744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.745744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.765282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:31.766659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:31.861100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 373/743] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:31.896090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403a25e3-631a-44bc-9689-9a783b630e49 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:31.899091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:31.945097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:31.946069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:31.946353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:31.946707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:31.947320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:31.948944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:31.951680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING keystone.server.flask.common [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 21 14:46:32.002426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:32.002426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:46:32.002426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [parameters: {'id': '2927ffe975224a2b9c781475fe173ca7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 21 14:46:32.002426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2927ffe975224a2b9c781475fe173ca7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2927ffe975224a2b9c781475fe173ca7. Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:32.010757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 14:46:32.012389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 21 14:46:32.013436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 14:46:32.014341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 14:46:32.017490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:32.018450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': '2927ffe975224a2b9c781475fe173ca7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:32.023747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:32.024556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2927ffe975224a2b9c781475fe173ca7. Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:32.025464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 371/744] 10.222.0.60 () {60 vars in 1140 bytes} [Fri Oct 21 14:46:31 2022] POST /identity/v3/projects => generated 262 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:32.026003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:32.109461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:32.126287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dd5dff-ce72-4e95-98f1-d43f4db252de None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:32.141918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:32.163909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:32.166834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:32.166834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:32.166834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:32.172988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:32.174791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:32.187628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:32.189197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:32.190096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:32.190522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 374/745] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:32.195822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:32.271060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de00192b-b26f-41e9-b413-8d65aba15022 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:32.314877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b275-b440-4fe7-901d-7bd7b4c338c4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:32.318569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:32.319226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:32.319509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:32.319892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:32.320579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:32.322172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75656820-736f-4be0-8794-4f4624dfedf4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:32.351136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:32.351746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:32.352027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:32.352299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:32.352961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:32.353266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:32.354751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975c7b1-ed58-4442-b201-92f9ab51d866 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:32.373004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 375/746] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:32 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:32.389577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 372/747] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:32 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:46:34.003776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098104-6ab8-4335-96c4-6c739b10ec60 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.004303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098104-6ab8-4335-96c4-6c739b10ec60 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.004303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5098104-6ab8-4335-96c4-6c739b10ec60 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.005881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 376/748] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:34.011169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8871-7640-41ac-88b1-c19416465492 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.011424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.011814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8871-7640-41ac-88b1-c19416465492 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.011814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d8871-7640-41ac-88b1-c19416465492 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.011947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.012033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.013137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 377/749] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:34.018242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.018525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.018833 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.027318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.035001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:34.036520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:34.038264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:34.039573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:34.054484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.122959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 378/750] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:34.124225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 373/751] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:34.396074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.402752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.508353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee07edf-d1fa-4552-9179-31093a8c409f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:34.535960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaf2f3-fdd2-4b3e-a438-ab7e740b8ba3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:34.547879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.556218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:34.557228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.557509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.562729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.562729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:34.562729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:34.573919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:34.575815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:34.577132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 379/752] 10.222.0.60 () {56 vars in 1126 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:34.582823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.585112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:34.585834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.586274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.586610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] PATH_INFO: `/v3/services/image` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.587408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:34.588848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:34.598812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:34.600821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:34.602068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 374/753] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services/image => generated 86 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:34.607946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.624864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.703354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa060a-f760-4c30-a468-c9f7b30a1ab5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:34.722500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7ee901-b538-4648-acb4-63a419e30eaf None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:34.736918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:34.737520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.737793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.738057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.738709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:34.739008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:34.740412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50e564c-b329-426f-99a5-f7f72fe6d039 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:34.745049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 380/754] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:34.746169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d751227-2c2f-4fe9-acd4-c5f35342cc30 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.746446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d751227-2c2f-4fe9-acd4-c5f35342cc30 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.746743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d751227-2c2f-4fe9-acd4-c5f35342cc30 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.751732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 381/755] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:34.751732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3d99-afde-4af2-98e7-c13cda5ea07f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.751732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3d99-afde-4af2-98e7-c13cda5ea07f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.751732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c3d99-afde-4af2-98e7-c13cda5ea07f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.751732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 382/756] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:34.753758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c3d99-afde-4af2-98e7-c13cda5ea07f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.761219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:34.761839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.762119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.762393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.763064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] RBAC: Adding query filter params (name=image) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:34.763365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:34.764756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a54681-a0a7-4178-8638-6f86a47b2a6e None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:34.769731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 375/757] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:34.770737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041661-a0ab-4bc9-97dd-d19a3320e01e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.771018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041661-a0ab-4bc9-97dd-d19a3320e01e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.771291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec041661-a0ab-4bc9-97dd-d19a3320e01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.788753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ec041661-a0ab-4bc9-97dd-d19a3320e01e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:34.789940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec041661-a0ab-4bc9-97dd-d19a3320e01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:34.811484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.854932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c3d99-afde-4af2-98e7-c13cda5ea07f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:34.857679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 376/758] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:34.858941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.859287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.860086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.881568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:34.888573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:34.890122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:34.890308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:34.891336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:34.891336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:34.891574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:34.892212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:34.892516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:34.893872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:34.898996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 383/759] 10.222.0.60 () {56 vars in 1122 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:34.899829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:34.968559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 377/760] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:34 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:34.970457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.028891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f29b66e-ae00-4745-8ded-144c68f8cea6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.059311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.075494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.076635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.077648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.078276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.079608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] RBAC: Adding query filter params (type=image) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:35.079920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.081324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.088327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 384/761] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:34 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:35.089990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9954e851-72c2-400c-80ba-54c5d551b5bb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.098186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.141550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.142175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.142452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.142756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.143425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.145341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.153837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.206793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e7efd-13fd-411e-9aad-cd8146741c51 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.229831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 378/762] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:35.239773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.253974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.255583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.255583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.255583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.256068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] RBAC: Adding query filter params (interface=public, service_id=4f722648288045109498df743a705b17, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:35.256380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.258582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.268162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 385/763] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/endpoints?service_id=4f722648288045109498df743a705b17&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:35.272322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.333741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.371253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49354f1-3c6f-4673-96c2-7ef4c83f5d36 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.393001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c9c5-afbb-49f0-bcdb-4a3145e397e6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.413308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.414010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.414287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.414551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.415158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.416632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:35.427563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:35.428605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.429473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 379/764] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:35.430348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.444844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.445463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.445748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.446011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.446616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.448014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.457992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 386/765] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/services => generated 1957 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:46:35.458856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.518397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.570704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64054a28-6d1c-4a75-909a-8ce81eff7a20 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.575761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf6e28-c87a-476f-8564-472094e8ba78 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.588192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.612102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.612719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.612992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.613257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.613902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.615372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.616376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.617000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.617279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.617550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.618189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:35.618492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.620175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.631670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 387/766] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:35.632671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.668554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 14:46:35.668554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 21 14:46:35.668554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [parameters: {'user_id': 'f3d837e5c75b40aea9cc6675a56cf80b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 21 14:46:35.668554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:35.678000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:35.678924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 14:46:35.679882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 14:46:35.680759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:35.681659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:35.682494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f3d837e5c75b40aea9cc6675a56cf80b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:35.683377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 21 14:46:35.685997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 380/767] 10.222.0.60 () {60 vars in 1131 bytes} [Fri Oct 21 14:46:35 2022] POST /identity/v3/users => generated 160 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:35.687314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.770870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d5f9a-0b6a-45db-a823-3ffc8e30f34a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.798975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f86c69-1376-4157-b717-baefe03891e2 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:35.819773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.820522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.820904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.821237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.824634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:35.824634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.824634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.826414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.833476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 388/768] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:35.840494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.845029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:35.847605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:35.847605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:35.847605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:35.855232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:35.857318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:35.870594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:35.877587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:35.878514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 381/769] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:35.882467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:35.897705 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:35.984543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5bee1-d678-4442-914d-b9efdc8bfab1 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:36.024408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329448c5-8a66-47a1-877f-fa62a6f75869 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:36.028662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:36.029391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:36.029920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:36.030106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:36.030945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] RBAC: Adding query filter params (interface=public, service_id=9bf5fe2504d94cb3901c60a8919a4e2b, region_id=RegionOne) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:36.031380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:36.033007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:36.038968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 389/770] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/endpoints?service_id=9bf5fe2504d94cb3901c60a8919a4e2b&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:36.046776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:36.068607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:36.069274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:36.069563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:36.069838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:36.070536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:36.070965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:36.072506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c204670f-ea4a-4e28-8c9b-84e352eb23af None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:36.114707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 382/771] 10.222.0.60 () {56 vars in 1101 bytes} [Fri Oct 21 14:46:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:36.173816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbe9753-daad-4a23-b249-1489034666eb None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:36.175898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:36.213751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:36.214390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:36.214997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:36.215531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:36.216459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:36.217876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74fe73-c93d-41ee-bbd8-c4f8a6edca63 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:36.225790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 390/772] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:46:36 2022] GET /identity/v3/services => generated 1957 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 14:46:37.468983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023016-fd09-4ff0-9f9b-24a3002f2da4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:37.469508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023016-fd09-4ff0-9f9b-24a3002f2da4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:37.469951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d023016-fd09-4ff0-9f9b-24a3002f2da4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:37.471514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 383/773] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:37.476755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57abad9-49d9-4ffd-95c9-89400468a46d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:37.477145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57abad9-49d9-4ffd-95c9-89400468a46d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:37.477306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57abad9-49d9-4ffd-95c9-89400468a46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:37.495498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b57abad9-49d9-4ffd-95c9-89400468a46d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:37.496688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b57abad9-49d9-4ffd-95c9-89400468a46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:37.564883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 391/774] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:37 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:37.758974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc3c0-f100-4b38-a10a-021c834cc61a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:37.759172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc3c0-f100-4b38-a10a-021c834cc61a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:37.759455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdc3c0-f100-4b38-a10a-021c834cc61a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:37.760811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 384/775] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:37.765961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb47-d251-4901-8024-02a874d78152 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:37.766253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb47-d251-4901-8024-02a874d78152 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:37.766531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789cb47-d251-4901-8024-02a874d78152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:37.784759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7789cb47-d251-4901-8024-02a874d78152 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:37.785949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7789cb47-d251-4901-8024-02a874d78152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:37.837422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdc3c0-f100-4b38-a10a-021c834cc61a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:37.854677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 392/776] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:37 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:37.886578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:37.946180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fdc3c0-f100-4b38-a10a-021c834cc61a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:37.986262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:37.986950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:37.987220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:37.987489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:37.988608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:37.990039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:37.994678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 385/777] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:37.997430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789cb47-d251-4901-8024-02a874d78152 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.025312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:38.098922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789cb47-d251-4901-8024-02a874d78152 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.133234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.133852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.134128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.134397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.135084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.136502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.139163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.239707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed861765-2213-48f6-8798-652b8d215dab None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.248931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:38.259041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 393/778] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:46:37 2022] POST /identity/v3/projects => generated 311 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:38.276258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.276904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.277110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.277377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] PATH_INFO: `/v3/services/image` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.277992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.279481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-3469990f-d800-427e-bd52-aee2622f41c5 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:38.290890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:38.292009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:38.295847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 386/779] 10.222.0.60 () {56 vars in 1105 bytes} [Fri Oct 21 14:46:38 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:38.299070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.385271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:38.409000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ca6c52-b9f6-4635-b330-bccd3f13af0d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.413538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915452a-1275-4f20-a715-7612301d0642 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.413854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915452a-1275-4f20-a715-7612301d0642 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.414132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915452a-1275-4f20-a715-7612301d0642 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.415856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 387/780] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:38.422836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079906-01ab-4e06-851b-3113b471406a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.423406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079906-01ab-4e06-851b-3113b471406a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.423569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71079906-01ab-4e06-851b-3113b471406a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.445449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-71079906-01ab-4e06-851b-3113b471406a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:38.447095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-71079906-01ab-4e06-851b-3113b471406a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:38.448002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.448691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.448974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.449304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.449971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] RBAC: Adding query filter params (name=image) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:38.450259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.451687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.459463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 394/781] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:38.464230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.527810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 388/782] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:38 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:38.571154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c519b69-1c00-4476-9e49-8fc11bf5886a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.606880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.607760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.608183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.608598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.609385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] RBAC: Adding query filter params (type=image) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:38.609861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.611442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.616797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 395/783] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:38.622386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71079906-01ab-4e06-851b-3113b471406a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.722764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71079906-01ab-4e06-851b-3113b471406a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.724391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:38.757272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.757882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.758159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.758428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.759139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.760726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.795459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:38.805508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:38.863822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 389/784] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:38 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:46:38.903375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb997d36-5e8e-400e-93a2-0140033bb941 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:38.955242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:38.956214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:38.956650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:38.957077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:38.957836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:38.959421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:38.971504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:38.972779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 21 14:46:38.973830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:38.976415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 396/785] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:38.982111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:39.094914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23023-7425-4798-9a2d-bd045c759ccf None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:39.130608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:39.131241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:39.131519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:39.131790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:39.132419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:39.132720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:39.134086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:39.139291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 390/786] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:39.145135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:39.212102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:39.247047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb068635-b81c-49f6-bfee-7c2c54caa400 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:39.284423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:39.285035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:39.285314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:39.285588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:39.286211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:39.286511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:39.287910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b9137-1abf-4a36-a8aa-a8f43ce82c34 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:39.297809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 397/787] 10.222.0.60 () {56 vars in 1112 bytes} [Fri Oct 21 14:46:39 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:39.299030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:39.324480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:39.402152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb73579-bc95-4a61-ad36-7e1580f4970b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:39.436998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:39.437652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:39.437934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:39.438286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:39.438922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:39.440290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44d2fc2-e6da-4cf7-9f45-8caa439dc509 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:39.542671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 391/788] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:46:39 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:40.581230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0683-9904-4488-8eda-46edc197e18d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:40.581750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0683-9904-4488-8eda-46edc197e18d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:40.581750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b0683-9904-4488-8eda-46edc197e18d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:40.583271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 398/789] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:40.588400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1579-8f1d-488d-baf7-5fadbdbfef53 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:40.588778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1579-8f1d-488d-baf7-5fadbdbfef53 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:40.588938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c1579-8f1d-488d-baf7-5fadbdbfef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:40.607059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8f6c1579-8f1d-488d-baf7-5fadbdbfef53 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:40.608234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8f6c1579-8f1d-488d-baf7-5fadbdbfef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:40.683560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 392/790] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:40.967447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b0683-9904-4488-8eda-46edc197e18d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:41.076543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b0683-9904-4488-8eda-46edc197e18d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:41.077350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ce56-7a3a-4715-9025-959488cdf7c6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.077643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ce56-7a3a-4715-9025-959488cdf7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.077922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2ce56-7a3a-4715-9025-959488cdf7c6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.079845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 393/791] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:41.084784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.085067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.085335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.092362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:41.105230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:41.106501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:41.107738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:41.115177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:41.115812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.116089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.116361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.117611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:41.119068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:41.120782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 399/792] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:41.126593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:41.177341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 394/793] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:41.235088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f89c42-d25a-43ee-a7a4-1d62941d10e1 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:41.268589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:41.269197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.269470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.269734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.270421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:41.272116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:41.399234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 400/794] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:46:41 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 14:46:41.447085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:41.555120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ee0f4-49ec-4d8c-947d-4f05d7a3fc3f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:41.592519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:41.593158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.593401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.593675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.595114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:41.596486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:41.600465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:41.637385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.sql.core [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:41.637385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:46:41.637385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [parameters: {'id': '8870bc1b576241618db1a5dafae729d8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 14:46:41.637385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81565) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b63f6807-8bb4-44bb-8f58-d4e00aec2d7f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:41.645945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 21 14:46:41.647333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application yield resource Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application next(self.gen) Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.rollback() Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.force_reraise() Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise self.value Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application session.commit() Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 21 14:46:41.648577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._prepare_impl() Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.session.flush() Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._flush(objects) Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application compat.raise_( Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application flush_context.execute() Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rec.execute(self) Oct 21 14:46:41.649460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 21 14:46:41.650355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result = self._query(query) Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application conn.query(q) Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application result.read() Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 21 14:46:41.651183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [parameters: {'id': '8870bc1b576241618db1a5dafae729d8', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 21 14:46:41.652049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:41.652835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:41.655178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 395/795] 10.222.0.60 () {60 vars in 1137 bytes} [Fri Oct 21 14:46:41 2022] POST /identity/v3/domains => generated 190 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:41.657138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:41.720827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:41.768179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98acff8-3788-4580-bd5f-6c00b016b5e5 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:41.786779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eacf5c-cb6f-4fca-b67b-ac79b03bf585 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.787087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eacf5c-cb6f-4fca-b67b-ac79b03bf585 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.787334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eacf5c-cb6f-4fca-b67b-ac79b03bf585 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.790671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 396/796] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:41.793746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.794137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.794479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.806476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:41.807082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:41.807354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:41.807625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:41.814836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:41.816044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:41.820668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:41.822118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:41.839130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:41.840098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:41.840974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:41.840974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:41.840974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:41.840974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 401/797] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:41.846309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:41.963056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc45f8-985d-47f1-9965-8a4fbf0c70fe None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:41.971305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:42.002506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:42.003620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:42.004258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:42.004534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:42.005544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:42.005842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:42.007245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:42.009865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 397/798] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:42.022667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 402/799] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:42.280777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:42.383947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59afe6-0519-4b9a-8989-53a3f666db52 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:42.410048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:42.417487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:42.418115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:42.418392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:42.418687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:42.429230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:42.434311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:42.435291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:42.436143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 398/800] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:46:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:42.441618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:42.539285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101d28f3-7d54-44fc-908a-ffdefa898beb None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:42.571679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:42.572277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:42.572551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:42.572820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:42.573465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:42.573762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:42.575170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:42.582817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 403/801] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Oct 21 14:46:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:42.588390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:42.685518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d57edf-8008-40b2-828a-96e07abbab35 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:42.718578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:42.719975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:42.720254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:42.720523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:42.730431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:42.731895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:42.746580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:42.747759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:42.748652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:42.748652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:42.748652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:42.748652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 399/802] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:42.753518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:42.782692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:42.852403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f1127-9dfa-47d0-a6e8-1c6d3bb50eab None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:42.884104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:42.884701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:42.884975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:42.885244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:42.885876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:42.886173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:42.887545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:42.899443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 404/803] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:42.905039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:42.945997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:43.003818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3254d-3f99-4da9-9ba7-9daac935ee31 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.036932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.037550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.037823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.038089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.046210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.047743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:43.058546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:43.059683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:43.060836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:43.064296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 400/804] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:46:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:43.065416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.153708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:43.165922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84ca0dc-0df8-4dbc-8b54-42f2b39f52f4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.200672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.201280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.201555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.201823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.202543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:43.202875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.204314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.221865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 405/805] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:46:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:43.227546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.323310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:43.326395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d5047e-6165-4eee-ac26-909f0d1b229b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.361045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.361670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.361951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.362221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.372104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.373529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:43.388013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:43.389137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:43.390254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:43.390254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:43.390254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:43.390779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 401/806] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:43.394780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.492130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671fc03c-cc2c-4a72-b306-9d0ef6141e1b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.525925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.526525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.526828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.527099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.527728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:43.528028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.529375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac070119-dbfa-46bf-b109-ae6d82a53b1b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.541239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 406/807] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:43.547153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.648743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab644467-e8b4-4e47-9965-c3babe1de7da None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.670092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3465-fb43-4af1-b1f4-bb5cd215f0d9 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.670378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3465-fb43-4af1-b1f4-bb5cd215f0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.670674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a3465-fb43-4af1-b1f4-bb5cd215f0d9 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.672047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 407/808] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:43.677441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.677720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.677990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.683431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.684044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.684331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.684587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.690907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:43.691248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.692676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.696822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:43.697997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:43.704505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:43.715925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 402/809] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:43.716818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.726951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:43.767905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 408/810] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:43.810011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a620214f-d2d1-4449-8935-26d0aaf34374 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.844387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.844917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.845194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.845462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:43.846140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:43.846441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:43.847916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:43.859257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 403/811] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:43.865698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:43.964034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f353a-24a3-41f5-a97f-2909b6d0b295 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:43.997254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:43.997912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:43.998188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:43.998459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/1535167c92354699abb73e06ca6802ab` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.001072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=9ee990610dd642059936212305d94fb6, role_id=1535167c92354699abb73e06ca6802ab)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:44.002500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:44.005441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:44.039493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:44.089626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 409/812] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:43 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/9ee990610dd642059936212305d94fb6/roles/1535167c92354699abb73e06ca6802ab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:44.110408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:44.144586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d49dfe3-2707-4d2b-b99d-e203a450bfa6 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:44.181142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:44.181794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:44.182072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:44.182345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.195407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:44.200782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:44.201821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:44.202712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 404/813] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:44.209527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:44.312480 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a9272-a59a-4637-bd55-d137a4bb854e None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:44.346895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:44.347507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:44.347788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:44.348061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.348721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:44.349023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:44.350393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:44.358471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 410/814] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:44.364736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:44.465658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250dd8d-7235-4e52-be1c-859278c09fcb None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:44.497654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:44.503029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:44.503636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:44.503911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:44.504185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] PATH_INFO: `/v3/users/5b3061d2f9bb44f9acbb828b92cca30d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.518233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] RBAC: Authorizing `identity:get_user(user_id=5b3061d2f9bb44f9acbb828b92cca30d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:44.519691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:44.521283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 405/815] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:44 2022] GET /identity/v3/users/5b3061d2f9bb44f9acbb828b92cca30d => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:44.527219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:44.585744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:44.628222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e1c49-15f5-48c7-b4e5-edb3511f79ea None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:44.661854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:44.662481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:44.662868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:44.663143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.664263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] RBAC: Authorizing `identity:get_project(project_id=76e7b609c821445684bb2569a1d73fe6)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:44.665688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807a7a01-b2e9-42b5-bcfb-073895166d3a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:44.667348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 411/816] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:46:44 2022] GET /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:44.673324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:44.771161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd586509-8618-4f0a-bdca-2593b1f91aca None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:44.804948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:44.805606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:44.805884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:44.806159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6/users/5b3061d2f9bb44f9acbb828b92cca30d/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:44.808107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=76e7b609c821445684bb2569a1d73fe6, user_id=5b3061d2f9bb44f9acbb828b92cca30d, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:44.809532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15767ca5-d3d1-41e4-8fbc-2d4edbdabcc7 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:44.926029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 406/817] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:44 2022] PUT /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6/users/5b3061d2f9bb44f9acbb828b92cca30d/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:46.236030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45a21-b28a-447d-8ada-e13b56a45eaa None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:46.236548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45a21-b28a-447d-8ada-e13b56a45eaa None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:46.236548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d45a21-b28a-447d-8ada-e13b56a45eaa None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:46.237813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 412/818] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:46.243044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:46.243469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:46.243567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:46.261811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:46.263070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:46.308273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:46.328694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 407/819] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:46.597053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45a21-b28a-447d-8ada-e13b56a45eaa None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:46.811212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d45a21-b28a-447d-8ada-e13b56a45eaa None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:46.836068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:46.844602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:46.845223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:46.848569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:46.848569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:46.855463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:46.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:46.861737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 21 14:46:46.862846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:46.866692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 413/820] 10.222.0.60 () {56 vars in 1120 bytes} [Fri Oct 21 14:46:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:46.867777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:46.971819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7bc52a-32e2-4d2d-b84c-3d5ee196bf28 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.005673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.006335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.006618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.006919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.007576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:47.007878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.009228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e994bc0-4113-4899-813f-52af42711fc2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.017110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 408/821] 10.222.0.60 () {56 vars in 1115 bytes} [Fri Oct 21 14:46:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:47.022997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.053246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2493b-6025-49bd-a3b7-b14d22c167ec None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.053536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2493b-6025-49bd-a3b7-b14d22c167ec None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.053815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2493b-6025-49bd-a3b7-b14d22c167ec None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.055175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 409/822] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:46:47.059881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.060162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.060437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.078713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:47.079926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:47.114686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.122912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dd6a68-8d52-4203-85c7-64ab08300d14 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.146676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 410/823] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:46:47.157008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.157629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.157905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.158171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.167920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.169362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:47.184192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:47.185313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:47.186205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:47.186205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:47.186205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:47.186205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 414/824] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:47.191139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.288318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca49022-5255-4df3-896c-b114ddebad65 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.320476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.321077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.321353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.321620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.322246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:47.322543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.323919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.335265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 411/825] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:47.340802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.379973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.418122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.439871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f8a08-eb9b-401b-9d54-a1deb88e3c80 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.448763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.473530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.474163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.474436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.474740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.481762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.483224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:46:47.494418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:47.495383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:47.496319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 415/826] 10.222.0.60 () {56 vars in 1093 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:47.501239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.518387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e9599-66fa-423a-8638-b542511594c3 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.552793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.553418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.553696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.553966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.564374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:46:47.569167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:47.570208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:46:47.571087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 412/827] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:47.576155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.597514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.601731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea0426-16bc-4022-bb09-f77f17a9ee97 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.634842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.635376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.635650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.635926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.636640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:47.636943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.638353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.655775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 416/828] 10.222.0.60 () {56 vars in 1133 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:47.661529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.667654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.676112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86d9e6-4c64-4e13-8f53-c794d11e4644 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.710083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.710708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.710989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.711257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.711909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:47.712208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.713557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.721591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 413/829] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:47.727369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.759117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c2d5c-a7b6-4918-b3b1-cde5d6288041 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.793007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.793607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.793886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.794152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.818306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.824957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2f17d1-94c8-43b3-be85-68c2953a5c56 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:47.858186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:47.858835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:47.859114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:47.859413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] PATH_INFO: `/v3/users/5b3061d2f9bb44f9acbb828b92cca30d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:47.859908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.860551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] RBAC: Authorizing `identity:get_user(user_id=5b3061d2f9bb44f9acbb828b92cca30d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:47.861344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.861962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:47.863566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 414/830] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/users/5b3061d2f9bb44f9acbb828b92cca30d => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:47.869188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.894451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:47.960875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 417/831] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 21 14:46:47.966532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:47.970229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef80f23-ccce-47a6-91c8-ba0de85bb69b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.003841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.004443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.004718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.004987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.021573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.064144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27f62e-01d6-44b4-9688-54a1b1a58e37 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.072136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.073586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.096718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.098559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.099222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.099502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.099775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.109845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.111282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:46:48.125542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:48.126483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:46:48.127458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:46:48.127458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:46:48.127458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:48.127458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 418/832] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 21 14:46:48.132493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.180469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 415/833] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 21 14:46:48.186332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.237177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c74dc-9d9e-4905-9581-529203578bdf None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.246184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.271930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.272648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.272987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.273841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.275029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:48.275485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.277405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.291862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 419/834] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:48.297510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.306176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831724dc-2244-45c3-91a2-8e01d3255fcd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.329650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.341417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.342196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.342574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.342973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.345010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] RBAC: Authorizing `identity:get_project(project_id=76e7b609c821445684bb2569a1d73fe6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.346549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.348278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 416/835] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:48.354327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.410747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17443dc6-87a5-4ff1-b9e0-f687d005fb39 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.447106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.447671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.447950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.448209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.456563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.457996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.462997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd4789-9c10-4926-b10d-30ee1326cefe None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:46:48.474053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:46:48.475032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:46:48.475928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 420/836] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:46:48.481021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.504121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.504721 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.504993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.505259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.505973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=76e7b609c821445684bb2569a1d73fe6, user.id=5b3061d2f9bb44f9acbb828b92cca30d) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:48.506282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.507730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.510267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.520779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 417/837] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/role_assignments?user.id=5b3061d2f9bb44f9acbb828b92cca30d&scope.project.id=76e7b609c821445684bb2569a1d73fe6&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:48.588077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0216d8e9-663a-4d65-af8e-b2a3705d9a83 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.622652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.623276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.623550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.623816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.624509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:48.624808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.626917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19545f6-8558-496b-bf8e-0ff8b41f9182 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.639883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 421/838] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:46:48.647385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:48.705173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:48.749092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cc8d5-72c8-47c3-ae36-451d4fd607ae None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:48.784536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:48.785137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:48.785416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:48.785683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:48.786393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] RBAC: Adding query filter params (role.id=1535167c92354699abb73e06ca6802ab, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=9ee990610dd642059936212305d94fb6) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:48.786733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:48.788150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e2b51-2624-4afa-a668-78f947b88f5d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:48.794031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 418/839] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:48 2022] GET /identity/v3/role_assignments?user.id=9ee990610dd642059936212305d94fb6&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=1535167c92354699abb73e06ca6802ab => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:50.697220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4a79-2459-48d2-aebe-e45067331414 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:50.697754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4a79-2459-48d2-aebe-e45067331414 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:50.697754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d4a79-2459-48d2-aebe-e45067331414 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:50.699105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 422/840] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:50.705143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:50.705541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:50.705692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:50.723774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:50.724951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:50.790639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 419/841] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:51.061558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4a79-2459-48d2-aebe-e45067331414 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:51.147305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:51.160353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d4a79-2459-48d2-aebe-e45067331414 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:51.193248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:51.193858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:51.194132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:51.194461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:51.195559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:51.196964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd210bfe-9817-4385-8701-5ea4399ee1f5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:51.198515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 423/842] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:46:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:51.204768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:51.251454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:51.302829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df4c-d1da-4b4a-b167-f87ae139e35d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:51.335601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:51.336197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:51.336473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:51.336744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:51.337381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:51.338820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb8a6c-cac2-4661-8afd-ecf4bdf6489b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:51.408544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 420/843] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:46:51 2022] POST /identity/v3/users => generated 286 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:46:53.597592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b9c7-797c-4840-b0c8-eca5d0623245 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:53.598126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b9c7-797c-4840-b0c8-eca5d0623245 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:53.598126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9b9c7-797c-4840-b0c8-eca5d0623245 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:53.599480 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 424/844] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:53.604649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:53.605080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:53.605196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:53.622922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:53.624101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:53.689115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 421/845] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:53.955662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b9c7-797c-4840-b0c8-eca5d0623245 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:54.055893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb9b9c7-797c-4840-b0c8-eca5d0623245 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:54.088903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:54.089518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:54.089803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:54.090069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] PATH_INFO: `/v3/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:54.091449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] RBAC: Authorizing `identity:get_role(role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:54.092797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:54.094556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 425/846] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:53 2022] GET /identity/v3/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:54.100372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:54.134114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:54.198154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba26ae-9a9f-450d-8d2f-abed7ec2d1fa None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:54.232178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:54.232738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:54.233014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:54.233285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] PATH_INFO: `/v3/users/00aa814997e44fa49ca4c4959859b2b6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:54.246421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] RBAC: Authorizing `identity:get_user(user_id=00aa814997e44fa49ca4c4959859b2b6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:54.247877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:54.249450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 422/847] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:54 2022] GET /identity/v3/users/00aa814997e44fa49ca4c4959859b2b6 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:54.256058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:54.280808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:54.352697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f63d89-3432-472b-977d-5aa555adac1f None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:54.386836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:54.387451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:54.387728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:54.387995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:54.389092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] RBAC: Authorizing `identity:get_project(project_id=76e7b609c821445684bb2569a1d73fe6)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:54.390523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98cf635-5f95-4e78-85f7-e6a79fbd836a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:54.392142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 426/848] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:46:54 2022] GET /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:54.397969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:54.487602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:54.495947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db644d4-2a2f-41a8-9b82-e2823dacccd1 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:54.528650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:54.529339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:54.529817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:54.530146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6/users/00aa814997e44fa49ca4c4959859b2b6/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:54.533351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] RBAC: Authorizing `identity:create_grant(project_id=76e7b609c821445684bb2569a1d73fe6, user_id=00aa814997e44fa49ca4c4959859b2b6, role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:54.534836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20c191-848d-4782-b286-2dddad24dfcd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:54.597414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 423/849] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:46:54 2022] PUT /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6/users/00aa814997e44fa49ca4c4959859b2b6/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:46:56.723346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c599e9-2602-492d-adfc-c1c2886ae9bc None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:56.723911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c599e9-2602-492d-adfc-c1c2886ae9bc None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:56.723911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c599e9-2602-492d-adfc-c1c2886ae9bc None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:56.725192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 427/850] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:46:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:46:56.730331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:56.730728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:56.730921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:56.749274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:46:56.750518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:46:56.815753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 424/851] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:46:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:46:57.086976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c599e9-2602-492d-adfc-c1c2886ae9bc None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:57.176861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:57.185972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c599e9-2602-492d-adfc-c1c2886ae9bc None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:57.218311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:57.219552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:57.219827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:57.220098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] PATH_INFO: `/v3/roles/e6c31a1ad1c74840b2b04e21d9ff67bf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:57.221485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] RBAC: Authorizing `identity:get_role(role_id=e6c31a1ad1c74840b2b04e21d9ff67bf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:57.222858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:57.224136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 428/852] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:57 2022] GET /identity/v3/roles/e6c31a1ad1c74840b2b04e21d9ff67bf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:57.229913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:57.309178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:57.326252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32d8fb-eade-435f-9d60-217a71151e9d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:57.359787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:57.360402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:57.360677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:57.360941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] PATH_INFO: `/v3/users/00aa814997e44fa49ca4c4959859b2b6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:57.362024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] RBAC: Authorizing `identity:get_user(user_id=00aa814997e44fa49ca4c4959859b2b6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:57.363463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:57.365001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 425/853] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:46:57 2022] GET /identity/v3/users/00aa814997e44fa49ca4c4959859b2b6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:57.371097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:57.468364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209935-99bf-4c2d-9469-86e798638e25 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:57.501818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:57.502336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:57.502609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:57.502899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:57.529456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:57.567906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:57.569330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:57.664049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 429/854] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:46:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:46:57.669998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:57.778438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0dd817-dbf5-4efb-a914-fe77c2a28832 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:57.811920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:57.813154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:57.813432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:57.813705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] PATH_INFO: `/v3/projects/76e7b609c821445684bb2569a1d73fe6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:57.814866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] RBAC: Authorizing `identity:get_project(project_id=76e7b609c821445684bb2569a1d73fe6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:57.816298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9842c947-aa9a-4d2f-b3d7-5edc29161827 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:57.817936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 426/855] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:46:57 2022] GET /identity/v3/projects/76e7b609c821445684bb2569a1d73fe6 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:46:57.824296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:46:57.887158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:46:57.931040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bee165-95b1-4cbf-b1e5-e7f43473c74b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:46:57.964382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:46:57.964978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:46:57.965252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:46:57.965522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:46:57.966229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] RBAC: Adding query filter params (role.id=e6c31a1ad1c74840b2b04e21d9ff67bf, scope.project.id=76e7b609c821445684bb2569a1d73fe6, user.id=00aa814997e44fa49ca4c4959859b2b6) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:46:57.966534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:46:57.967974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9479239-1d58-483a-8044-0ab6a697c3e8 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:46:57.974086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 430/856] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:46:57 2022] GET /identity/v3/role_assignments?user.id=00aa814997e44fa49ca4c4959859b2b6&scope.project.id=76e7b609c821445684bb2569a1d73fe6&role.id=e6c31a1ad1c74840b2b04e21d9ff67bf => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:00.185700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2294-799a-4d1a-8102-cd1381437f8e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:00.186245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2294-799a-4d1a-8102-cd1381437f8e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:00.186245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e2294-799a-4d1a-8102-cd1381437f8e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:00.188398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 427/857] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:00.193582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:00.193964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:00.194123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:00.211945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:00.213113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:00.278119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 431/858] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:00.552791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e2294-799a-4d1a-8102-cd1381437f8e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:00.571844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:00.671048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e2294-799a-4d1a-8102-cd1381437f8e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:00.704875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:00.705504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:00.705765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:00.706037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:00.707158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:00.708565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ad805-a90f-457d-8f00-3b3b3477256a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:00.710086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 428/859] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:47:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:00.715955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:00.770186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:00.815086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab25fad9-4fc3-472e-a69e-a140c3f0c96a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:00.857464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:00.858078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:00.858358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:00.858660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:00.860028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:00.861454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36535b-1238-4e99-9aa0-a29d60abe5a3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:00.940805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 432/860] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:47:00 2022] POST /identity/v3/projects => generated 311 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:47:03.152066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167201c-fd15-432b-89cc-1e5114357e1f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:03.152596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167201c-fd15-432b-89cc-1e5114357e1f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:03.152703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6167201c-fd15-432b-89cc-1e5114357e1f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:03.153991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 429/861] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:03.159311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:03.159688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:03.159859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:03.178367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:03.179565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:03.243589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 433/862] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:03.513048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167201c-fd15-432b-89cc-1e5114357e1f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:03.597684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:03.613639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6167201c-fd15-432b-89cc-1e5114357e1f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:03.647177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:03.647786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:03.648061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:03.648576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:03.649678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:03.651137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047380e-aa46-4212-9b41-33f8c417dae5 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:03.652683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 430/863] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:47:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:03.659268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:03.693238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:03.757396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79be5c-6133-4704-a39e-97bdfb6991de None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:03.791239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:03.791774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:03.792053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:03.792325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:03.792983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:03.794401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e5967-074f-44e3-b0f2-89108142e9b9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:03.877798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 434/864] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:47:03 2022] POST /identity/v3/users => generated 286 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:47:06.041111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad016-f6bc-435d-9215-064c09252ef3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.041597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad016-f6bc-435d-9215-064c09252ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.041716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fad016-f6bc-435d-9215-064c09252ef3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.043032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 431/865] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:06.048675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.049138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.049240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.067254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:06.068417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:06.131167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 435/866] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:06.404816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad016-f6bc-435d-9215-064c09252ef3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:06.505005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fad016-f6bc-435d-9215-064c09252ef3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:06.538761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:06.539380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.539657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.539927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.550325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:47:06.555034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:47:06.556096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:47:06.557120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 432/867] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:47:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:47:06.561974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:06.584771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:06.655909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f932c-a900-451c-96c3-51ff4297a498 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:06.689125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:06.689705 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.689978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.690244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.690919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:06.691213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:06.692614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:06.700550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 436/868] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:06.706143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:06.722311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:06.803334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5630b1-d439-468e-91a9-e57ca3151280 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:06.836459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:06.837074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.837352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.837620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] PATH_INFO: `/v3/users/6a9babd1810042a8a2ea9e2d6bd96839` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.850652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] RBAC: Authorizing `identity:get_user(user_id=6a9babd1810042a8a2ea9e2d6bd96839)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:06.852076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:06.853725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 433/869] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:47:06 2022] GET /identity/v3/users/6a9babd1810042a8a2ea9e2d6bd96839 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:06.859587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:06.940645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:06.956222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60a6f-d844-4a9e-813d-26f688e10547 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:06.989182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:06.989806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:06.990079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:06.990347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] PATH_INFO: `/v3/projects/212db1c267954fdfb49b1aa4741e7e64` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:06.991465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=212db1c267954fdfb49b1aa4741e7e64)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:06.992875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ffff0a-9d83-4777-9625-2b685d174ef4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:06.994471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 437/870] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:47:06 2022] GET /identity/v3/projects/212db1c267954fdfb49b1aa4741e7e64 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:47:07.000689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:07.068768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:07.099203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ede31c-f468-4f44-9bcc-731a1a1a5e82 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:07.132243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:07.132890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:07.133167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:07.133439 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] PATH_INFO: `/v3/projects/212db1c267954fdfb49b1aa4741e7e64/users/6a9babd1810042a8a2ea9e2d6bd96839/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:07.135305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] RBAC: Authorizing `identity:create_grant(project_id=212db1c267954fdfb49b1aa4741e7e64, user_id=6a9babd1810042a8a2ea9e2d6bd96839, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:07.136722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047a0adc-706a-43c6-980e-8691b063da4d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:07.200834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 434/871] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:47:06 2022] PUT /identity/v3/projects/212db1c267954fdfb49b1aa4741e7e64/users/6a9babd1810042a8a2ea9e2d6bd96839/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:47:09.328856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415c32b-b039-4291-97a8-db3cc717874d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:09.329397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415c32b-b039-4291-97a8-db3cc717874d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:09.329397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415c32b-b039-4291-97a8-db3cc717874d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:09.330650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 438/872] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:09.335974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:09.336362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:09.336513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:09.354879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:09.356057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:09.420798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 435/873] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:09.691187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415c32b-b039-4291-97a8-db3cc717874d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:09.790190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415c32b-b039-4291-97a8-db3cc717874d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:09.821946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:09.822568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:09.822870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:09.835605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:09.835605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:47:09.838683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:47:09.839753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:47:09.840626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 439/874] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:47:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:47:09.846276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:09.905148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:09.942696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d80df-5d4f-46c8-97de-cccb2f5308a2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:09.977238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:09.977846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:09.978117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:09.978383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:09.979423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:09.979423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:09.980696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:09.989222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 436/875] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:47:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:09.995094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:10.009815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:10.091527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824a553-5064-49c7-a11a-937e9aae01a6 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:10.123666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:10.124280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:10.124560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:10.125461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] PATH_INFO: `/v3/users/6a9babd1810042a8a2ea9e2d6bd96839` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:10.126651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] RBAC: Authorizing `identity:get_user(user_id=6a9babd1810042a8a2ea9e2d6bd96839)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:10.128065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:10.129604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 440/876] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:47:09 2022] GET /identity/v3/users/6a9babd1810042a8a2ea9e2d6bd96839 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:10.135311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:10.231247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258317-af5e-4955-9fef-e94e57de4e7c None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:10.253440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:10.263482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:10.264171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:10.264447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:10.264715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:10.329709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:10.331159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:10.425438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 437/877] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:47:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:47:10.431294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:10.509932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:10.527996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1c930-a9b3-4927-a4da-c16743419d02 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:10.560288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:10.560899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:10.561175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:10.561445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] PATH_INFO: `/v3/projects/212db1c267954fdfb49b1aa4741e7e64` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:10.563287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] RBAC: Authorizing `identity:get_project(project_id=212db1c267954fdfb49b1aa4741e7e64)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:10.564698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f7f4bb-d486-4e05-946a-701bfdd74459 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:10.566289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 441/878] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:47:10 2022] GET /identity/v3/projects/212db1c267954fdfb49b1aa4741e7e64 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:10.572415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:10.596463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:10.668887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f645883-2232-4d13-8eb1-d0a44bcdd78a None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:10.702771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:10.703493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:10.703768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:10.704040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:10.704773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=212db1c267954fdfb49b1aa4741e7e64, user.id=6a9babd1810042a8a2ea9e2d6bd96839) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:10.705065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:10.706483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2cff7e-9ff3-4f36-8d5e-213d6b620c31 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:10.712179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 438/879] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:47:10 2022] GET /identity/v3/role_assignments?user.id=6a9babd1810042a8a2ea9e2d6bd96839&scope.project.id=212db1c267954fdfb49b1aa4741e7e64&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:12.848437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08327e-5da3-4dbf-a807-7bc34058dd6e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:12.848966 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08327e-5da3-4dbf-a807-7bc34058dd6e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:12.848966 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08327e-5da3-4dbf-a807-7bc34058dd6e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:12.850296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 442/880] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:12.855763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3be51-0105-4a8b-aa4b-f3abb91c4011 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:12.858760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3be51-0105-4a8b-aa4b-f3abb91c4011 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:12.858760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3be51-0105-4a8b-aa4b-f3abb91c4011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:12.874036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-62b3be51-0105-4a8b-aa4b-f3abb91c4011 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:12.875265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-62b3be51-0105-4a8b-aa4b-f3abb91c4011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:12.939582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 439/881] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 21 14:47:13.207943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08327e-5da3-4dbf-a807-7bc34058dd6e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:13.304677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08327e-5da3-4dbf-a807-7bc34058dd6e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:13.337483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:13.338073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:13.338350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:13.338620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:13.339309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:13.340655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eef11c9-ef25-42a2-a873-8d4ae74db8a8 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:13.420250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 443/882] 10.222.0.60 () {60 vars in 1138 bytes} [Fri Oct 21 14:47:13 2022] POST /identity/v3/domains => generated 267 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:47:15.578393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c231-298b-4748-a4ab-f758665003b5 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:15.579055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c231-298b-4748-a4ab-f758665003b5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:15.579055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd7c231-298b-4748-a4ab-f758665003b5 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:15.580279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 440/883] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:15.585668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:15.586147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:15.586277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:15.605249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:15.606427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:15.608397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:15.670957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 444/884] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:15.939857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7c231-298b-4748-a4ab-f758665003b5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:15.959174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:16.042877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd7c231-298b-4748-a4ab-f758665003b5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:16.076114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:16.076745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:16.077025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:16.077295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] PATH_INFO: `/v3/domains/a95724820c224de8988f5f471795ec3b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:16.088237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a95724820c224de8988f5f471795ec3b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:16.089673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e98f79-e5eb-4d17-9790-41761bbc1008 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:16.091309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 441/885] 10.222.0.60 () {56 vars in 1183 bytes} [Fri Oct 21 14:47:15 2022] GET /identity/v3/domains/a95724820c224de8988f5f471795ec3b => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:16.098354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:16.213136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a477542a-85ce-484d-b8fc-4d17cd7160d8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:16.247545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:16.248150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:16.248427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:16.248698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:16.249392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:16.251440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf01c3-f6f9-4a14-b8ad-f59f9b40df6c None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:16.260805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:16.360284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 445/886] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:47:16 2022] POST /identity/v3/projects => generated 361 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:47:18.546078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f237b5-4cf1-4410-b0e8-79d6f7c18a17 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:18.547303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f237b5-4cf1-4410-b0e8-79d6f7c18a17 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:18.547303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f237b5-4cf1-4410-b0e8-79d6f7c18a17 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:18.548628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 442/887] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:18.554153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:18.554576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:18.554715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:18.572185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:18.573345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:18.636655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 446/888] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:18.904768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f237b5-4cf1-4410-b0e8-79d6f7c18a17 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:18.984056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:19.005849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f237b5-4cf1-4410-b0e8-79d6f7c18a17 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:19.040293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:19.040943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:19.041223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:19.041498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] PATH_INFO: `/v3/domains/a95724820c224de8988f5f471795ec3b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:19.042611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=a95724820c224de8988f5f471795ec3b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:19.044067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6ed291-fac5-4ad0-ab87-ad632666bb06 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:19.045716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 443/889] 10.222.0.60 () {56 vars in 1183 bytes} [Fri Oct 21 14:47:18 2022] GET /identity/v3/domains/a95724820c224de8988f5f471795ec3b => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:19.051633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:19.146061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab60621-553a-4140-81fb-919cb0b03b02 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:19.165220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:19.178183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:19.183637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:19.183637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:19.183637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:19.183637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:19.183637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c875920c-d785-400a-aca8-68a3a48b0862 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:19.258988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 447/890] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:47:19 2022] POST /identity/v3/users => generated 311 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:47:21.466086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3df83-6719-4ec4-be8c-c686cd5b6820 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:21.466646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3df83-6719-4ec4-be8c-c686cd5b6820 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:21.466646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3df83-6719-4ec4-be8c-c686cd5b6820 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:21.468003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 444/891] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:21.474193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:21.474608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:21.474778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:21.493400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:21.494568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:21.559961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 448/892] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:21.833159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3df83-6719-4ec4-be8c-c686cd5b6820 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:21.990853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3df83-6719-4ec4-be8c-c686cd5b6820 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:22.023952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:22.024566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:22.024842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:22.025112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:22.035437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:47:22.040150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:47:22.041312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:47:22.042349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 445/893] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:47:22.047690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:22.147335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dac0a9-72b9-4b8a-9879-b33d466b9ea3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:22.154418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:22.180354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:22.180944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:22.181219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:22.181483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:22.182128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:22.182424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:22.183836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:22.192783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 449/894] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:47:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:22.198550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:22.209349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:22.298798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b47417c-d45a-4c9a-80ff-dd0dfbb1c614 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:22.332676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:22.333281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:22.333559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:22.333823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] PATH_INFO: `/v3/users/11a5f58c0d6c408681b986705a08d218` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:22.346767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] RBAC: Authorizing `identity:get_user(user_id=11a5f58c0d6c408681b986705a08d218)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:22.348176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:22.349729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 446/895] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:47:22 2022] GET /identity/v3/users/11a5f58c0d6c408681b986705a08d218 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:22.355616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:22.451056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab453622-637e-4b5e-b72d-58ccf75d1f66 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:22.483773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] PATH_INFO: `/v3/projects/a11f5ff8894940ab8ca623b8a22cf577` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] RBAC: Authorizing `identity:get_project(project_id=a11f5ff8894940ab8ca623b8a22cf577)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1948c63e-6e3d-404b-ae66-45f9e67428a7 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:22.490148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 450/896] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:47:22 2022] GET /identity/v3/projects/a11f5ff8894940ab8ca623b8a22cf577 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:22.495912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:22.556850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:22.592629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720893a-58a0-4df9-97f6-6c756589d457 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:22.626915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:22.627555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:22.627830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:22.628097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] PATH_INFO: `/v3/projects/a11f5ff8894940ab8ca623b8a22cf577/users/11a5f58c0d6c408681b986705a08d218/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:22.629955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] RBAC: Authorizing `identity:create_grant(project_id=a11f5ff8894940ab8ca623b8a22cf577, user_id=11a5f58c0d6c408681b986705a08d218, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:22.631457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e1392-f16e-4262-be56-991ebb7ae914 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:22.722339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 447/897] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:47:22 2022] PUT /identity/v3/projects/a11f5ff8894940ab8ca623b8a22cf577/users/11a5f58c0d6c408681b986705a08d218/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:47:24.826267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4151a-e03a-456f-add7-42d2309e7b8a None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:24.826860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4151a-e03a-456f-add7-42d2309e7b8a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:24.826860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4151a-e03a-456f-add7-42d2309e7b8a None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:24.828770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 451/898] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:47:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:47:24.834173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:24.834589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:24.834823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:24.853146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:47:24.854330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:47:24.919499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 448/899] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:47:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:47:25.193513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4151a-e03a-456f-add7-42d2309e7b8a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:25.229706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:25.293019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4151a-e03a-456f-add7-42d2309e7b8a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:25.325259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:25.325870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:25.326145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:25.326413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:25.336608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:47:25.341299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:47:25.342389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:47:25.343974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 452/900] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:47:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:47:25.349193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:25.402156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:25.445481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4651d22-33ba-474b-a66b-ecdf0e6b968f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:25.478509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:25.479139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:25.479414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:25.479686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:25.480337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:25.480639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:25.481981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:25.490361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 449/901] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:47:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:25.496240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:25.586880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:25.592217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec6ae6-6289-45a6-9833-6b0d5874520d None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:25.624412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:25.625193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:25.625474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:25.625743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] PATH_INFO: `/v3/users/11a5f58c0d6c408681b986705a08d218` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:25.627003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] RBAC: Authorizing `identity:get_user(user_id=11a5f58c0d6c408681b986705a08d218)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:25.628414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:25.629944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 453/902] 10.222.0.60 () {56 vars in 1177 bytes} [Fri Oct 21 14:47:25 2022] GET /identity/v3/users/11a5f58c0d6c408681b986705a08d218 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:25.635676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:25.733066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906b7e3-98ae-4480-8392-fc38e764b356 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:25.748273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:25.766002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:25.766595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:25.766901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:25.767175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:25.832808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:25.834230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:25.929203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 450/903] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:47:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:47:25.935090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:26.032312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a2606-f041-48de-97ef-6b6394900226 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:26.063979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:26.064588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:26.064864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:26.065130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] PATH_INFO: `/v3/projects/a11f5ff8894940ab8ca623b8a22cf577` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:26.066217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] RBAC: Authorizing `identity:get_project(project_id=a11f5ff8894940ab8ca623b8a22cf577)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:26.067654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f54c8-aa38-4652-9380-de41fdee462f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:26.069236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 454/904] 10.222.0.60 () {56 vars in 1186 bytes} [Fri Oct 21 14:47:25 2022] GET /identity/v3/projects/a11f5ff8894940ab8ca623b8a22cf577 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:26.075375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:47:26.092522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:26.171795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8216865-a738-49c6-a5ac-54cab9437454 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:47:26.205243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:47:26.205835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:47:26.206114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:47:26.206388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:47:26.207134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=a11f5ff8894940ab8ca623b8a22cf577, user.id=11a5f58c0d6c408681b986705a08d218) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:47:26.207442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:47:26.208859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fbee9-a6bd-4999-a836-8021d8e3a087 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:47:26.214459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 451/905] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:47:26 2022] GET /identity/v3/role_assignments?user.id=11a5f58c0d6c408681b986705a08d218&scope.project.id=a11f5ff8894940ab8ca623b8a22cf577&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:47:36.059347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=84 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:47:36.211933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=60 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:41.350487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ab3c-6b2d-4405-bf49-72bcfa9a5eb4 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:41.351070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ab3c-6b2d-4405-bf49-72bcfa9a5eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:41.351070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e59ab3c-6b2d-4405-bf49-72bcfa9a5eb4 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:41.352298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 455/906] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:48:41.357505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:41.357898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:41.358077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:41.376597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:48:41.377833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:48:41.443315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 452/907] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:48:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:48:41.711928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59ab3c-6b2d-4405-bf49-72bcfa9a5eb4 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:41.813107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e59ab3c-6b2d-4405-bf49-72bcfa9a5eb4 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:41.845944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:41.846563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:41.846902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:41.847170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:41.857410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:41.858867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:41.873232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:41.874259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:41.875601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:41.875601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:41.875601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:41.875601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 456/908] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:41.881590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:41.979011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45179ec-1e12-4553-8217-e4f0074131f4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:42.007826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:42.013563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:42.014170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:42.014449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:42.014754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:42.015392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:42.015696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:42.017092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb5d3-a534-44b4-bd2d-a9b2d5eca6d2 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:42.028489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 453/909] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:42.034117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:42.087019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:42.133940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb881e-7b76-4c23-bc0b-85e5792bc0e3 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:42.166981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:42.167582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:42.167860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:42.168129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:42.168765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:42.170182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981031a3-80fe-41c8-89b1-823e59e3cf3b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:42.360711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 457/910] 10.222.0.60 () {60 vars in 1132 bytes} [Fri Oct 21 14:48:42 2022] POST /identity/v3/users => generated 251 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:48:44.503140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc010e-d479-4649-8dae-cb404e88816e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:44.503697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc010e-d479-4649-8dae-cb404e88816e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:44.503697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fc010e-d479-4649-8dae-cb404e88816e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:44.505015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 454/911] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:48:44.510900 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:44.511342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:44.511436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:44.530516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:48:44.531774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:48:44.596881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 458/912] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:48:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:48:44.869118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc010e-d479-4649-8dae-cb404e88816e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:44.970869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fc010e-d479-4649-8dae-cb404e88816e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.005882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.006578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.006881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.007152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.017092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:48:45.021958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:45.023192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:45.024067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 455/913] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:48:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:45.029496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.069997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:45.128846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbed215-3eb4-4c53-a343-ec0bbbf46417 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.163453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.164107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.164384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.164651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.165305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] RBAC: Adding query filter params (name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:45.165603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.166985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.174598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 459/914] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:45.180996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.231093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:45.279782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55359ac-4680-4920-b017-056efc1a5025 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.312372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.312997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.313277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.313558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.323830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.325271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:45.339986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:45.341178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:45.342459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:45.342459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:45.342459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:45.342459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 456/915] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:45.346935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.445463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:45.448302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89138ad6-6ab1-467d-9834-6db1aee12040 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.480121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.480716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.480990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.481253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.481879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:45.482176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.483539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.495354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 460/916] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:45.501092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.602166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b7f4a-1fc0-43e5-bd83-7879e168d8aa None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.603688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:45.636666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.637301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.637578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.637851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.644860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.646290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:48:45.658244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:45.659322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:45.660311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 457/917] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:45.666063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.762505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c13fd4-dd10-45e9-a8ca-cc103625db28 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.795393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.795996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.796271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.796540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.797254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:45.797558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.799008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.815891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 461/918] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:45.821823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:45.922093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21bce42-c931-41e1-851f-9c5fbda0ed75 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:45.956049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:45.956677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:45.956973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:45.957229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:45.967840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:45.969275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:45.984431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:45.986820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:45.987917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:45.987917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:45.987917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:45.987917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 458/919] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:45.991892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:46.011020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:46.093181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd42bd-92f5-4607-9331-fd277ca65c0b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:46.127424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:46.128028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:46.128315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:46.128588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:46.129284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:46.129590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:46.131201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:46.142443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 462/920] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:46.148755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:46.165914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:46.248415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a4bbc-226a-4e6e-90c1-eb5e0572a981 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:46.282763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:46.283383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:46.283660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:46.283930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:46.291172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:46.292597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:48:46.303468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:46.307723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:46.308829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 459/921] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:46.310429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:46.388756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:46.410234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e259f51-cbed-4ff4-ad38-e0577b30e23a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:46.443997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:46.444389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:46.444666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:46.444980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:46.445666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:46.445969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:46.447410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c909862-c06b-4d5c-aaea-a28aeace2c52 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:46.459678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 463/922] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:46.465638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:46.540749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:46.564242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20f8bd-1cab-4b8f-802b-91e651e4f40d None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:46.597253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:46.597904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:46.598182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:46.598457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/e05f107798a24fa9a0e65918076f2cc7/roles/c768011f94c9455fb0c0b7765f5a02a5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:46.613268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=e05f107798a24fa9a0e65918076f2cc7, role_id=c768011f94c9455fb0c0b7765f5a02a5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:46.614737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594b5db-da51-447c-a89e-39052a2e66bf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:46.697675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 460/923] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:48:46 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/e05f107798a24fa9a0e65918076f2cc7/roles/c768011f94c9455fb0c0b7765f5a02a5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:48:48.824427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3f35-626f-439a-a339-23dfa63a285f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:48.825091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3f35-626f-439a-a339-23dfa63a285f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:48.825091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcf3f35-626f-439a-a339-23dfa63a285f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:48.826304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 464/924] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:48:48.831463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:48.831886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:48.832029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:48.850908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:48:48.862782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:48:48.919611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 461/925] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:48:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:48:49.194806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf3f35-626f-439a-a339-23dfa63a285f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:49.295000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcf3f35-626f-439a-a339-23dfa63a285f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:49.328690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:49.329316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:49.329592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:49.329881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:49.336665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:49.340399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:48:49.345143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:49.353525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:49.354645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 465/926] 10.222.0.60 () {56 vars in 1102 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:49.355410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:49.406846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:49.450770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2139bf-519b-4de5-9bfb-018016740d9d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:49.484316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:49.485001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:49.485280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:49.485555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:49.486206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:49.486509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:49.487895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:49.496315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 462/927] 10.222.0.60 () {56 vars in 1103 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:49.502887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:49.600674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ff78-0b26-4b2f-a979-aad54b2f29c9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:49.633619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:49.634235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:49.634508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:49.634808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:49.644343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:49.645770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:49.660244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:49.661321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:49.662997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:49.662997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:49.662997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:49.662997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 466/928] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:49.668025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:49.764326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93701-da3c-4195-9868-28ad8a4b3ce4 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:49.790638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:49.798916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:49.800050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:49.800050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:49.800050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:49.800690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:49.800989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:49.802371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:49.814554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 463/929] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:49.820291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:49.869534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:49.918434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8a042-aea9-4740-a534-ba3d59e36410 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:49.951946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:49.952577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:49.952912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:49.953312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:49.961399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:49.962856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:48:49.975109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:49.976093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:49.977102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 467/930] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:49.982343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.080770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607ef625-c400-4ad9-b28e-0c2c72b0b814 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:50.114182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:50.114845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:50.115393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:50.115526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:50.116236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:50.116590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:50.117992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:50.134503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 464/931] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Oct 21 14:48:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:50.140395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.239318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5647c41-cdfd-4878-af81-0ccbb36a2246 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:50.249198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:50.272257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:50.272835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:50.273106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:50.273372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:50.337785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:50.339198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:50.434373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 468/932] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:48:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:48:50.440435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.475329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:50.538566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49f3d6-b8bb-4238-ad44-3fee22d911b8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:50.571082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:50.571697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:50.571970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:50.572236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:50.581661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:50.583167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:50.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:50.598931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:50.600209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:50.600209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:50.600209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:50.600209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 465/933] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:50.604830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.616589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:50.704713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5bec8-7648-4b15-8601-828a70c16e2a None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:50.738645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:50.739249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:50.739521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:50.739806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:50.740913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:50.741214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:50.742564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:50.753243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 469/934] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:50.759561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.844391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:50.859327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd1c30-efa1-42b4-9121-8d2898327d13 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:50.892770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:50.893392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:50.893669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:50.893940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:50.900884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:50.902446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:48:50.914319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:50.919171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:50.920358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 466/935] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:48:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:50.921401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:50.993229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:51.023145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc45ce9-c667-4ca7-8bc9-3c704841025b None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:51.055747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:51.056356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:51.056763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:51.057048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:51.057737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:51.058039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:51.059478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a3cda-7b19-4979-9c33-ef3057e3e81a None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:51.071157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 470/936] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:48:51.077521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:51.180469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0609610-3003-48de-8df4-8721753d8832 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:51.215158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:51.215752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:51.216031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:51.216300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:51.217016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] RBAC: Adding query filter params (role.id=c768011f94c9455fb0c0b7765f5a02a5, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=e05f107798a24fa9a0e65918076f2cc7) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:51.217321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:51.218789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308b6a7-6cad-40c9-af96-2a5d56001d35 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:51.225082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 467/937] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:48:51 2022] GET /identity/v3/role_assignments?user.id=e05f107798a24fa9a0e65918076f2cc7&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=c768011f94c9455fb0c0b7765f5a02a5 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:53.413267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b837f-9086-40d4-8310-eff99d6045eb None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:53.413767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b837f-9086-40d4-8310-eff99d6045eb None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:53.413767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b837f-9086-40d4-8310-eff99d6045eb None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:53.415100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 471/938] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:48:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:48:53.420297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:53.420799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:53.420877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:53.438957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:48:53.440129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:48:53.442918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:53.505130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 468/939] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:48:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:48:53.772932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b837f-9086-40d4-8310-eff99d6045eb None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:53.871753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b837f-9086-40d4-8310-eff99d6045eb None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:53.900588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:53.906135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:53.906789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:53.907077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:53.907350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:53.917227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:48:53.922085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:53.923205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:53.924248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 472/940] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:53.929966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.030611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2fb0f-c87a-4d16-98c3-fc51e684129b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.064452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.065057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.065347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.065616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.066273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:54.066569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.068596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.076717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 469/941] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:54.082496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.184569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562aeb5-7b2e-4873-96c6-c8d37912ce52 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.218321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.219051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.219381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.219647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.230352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.231813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:54.246283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:54.247492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:54.248585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:54.248585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:54.248585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:54.248585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 473/942] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:54.253359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.256281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:54.354076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b3d9b5-5d5e-42a7-ade6-b1842b00aa42 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.388189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.388804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.389080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.389352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.389987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:54.390292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.391688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.404229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 470/943] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:54.410013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.453506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:54.508971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad06bf0e-733f-4016-8afd-e73193f710fd None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.543066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.543683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.543958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.544224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.551328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.552734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:48:54.563715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:54.568629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:54.569644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 474/944] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:54.571573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.632422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:54.669699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815b5ec-bbc7-44b2-aec9-22bc9322db87 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.703294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.703890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.704162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.704429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.705144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:54.705440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.706873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.723536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 471/945] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 14:48:54.729497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.830855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a584f2-eebc-4c39-a921-0ab7b91ca270 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:54.832363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:54.863924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:54.864551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:54.865362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:54.865679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:54.875762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:54.877175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:54.892019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:54.893103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:54.894176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:54.894176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:54.894176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:54.894541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 475/946] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:54.899132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:54.998580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1600bc-ba86-4e10-9177-fe3f606aa8ee None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:55.019726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:55.033722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:55.034326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:55.034605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:55.044420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:55.044420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:55.044420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:55.044420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:55.048792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 472/947] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:55.055152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:55.153683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda3407-c804-4a1e-8a29-4b752229a6f4 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:55.187242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:55.187951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:55.188287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:55.188562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:55.195192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:55.196610 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:48:55.208255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:55.209380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:55.210460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 476/948] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:48:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:55.215490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:55.314065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4342793-eb35-4fc4-a72c-72581ded39ac None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:55.346986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:55.347588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:55.347887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:55.348155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:55.348834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:55.350078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:55.351536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3409053b-3fc8-4745-9586-6c897eccc9fc None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:55.363166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 473/949] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:48:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:55.369829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:55.386154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:55.468784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dad70-7ddc-4fc1-bd64-b5b5892fce95 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:55.503251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:55.503898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:55.504170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:55.504436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] PATH_INFO: `/v3/projects/d59dd3ce7504494bb481758b4195d883/users/e05f107798a24fa9a0e65918076f2cc7/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:55.506727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] RBAC: Authorizing `identity:create_grant(project_id=d59dd3ce7504494bb481758b4195d883, user_id=e05f107798a24fa9a0e65918076f2cc7, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:55.508178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67fcb8-5488-4e6c-848d-0b2751f4859f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:55.635152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 477/950] 10.222.0.60 () {58 vars in 1439 bytes} [Fri Oct 21 14:48:55 2022] PUT /identity/v3/projects/d59dd3ce7504494bb481758b4195d883/users/e05f107798a24fa9a0e65918076f2cc7/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 14:48:57.765883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797624-d94f-44d7-b553-de020e88b3c3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:57.766435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797624-d94f-44d7-b553-de020e88b3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:57.766435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85797624-d94f-44d7-b553-de020e88b3c3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:57.767799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 474/951] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:48:57.773179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:57.773561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:57.773728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:57.792637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:48:57.793824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:48:57.858159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 478/952] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:48:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:48:58.128777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797624-d94f-44d7-b553-de020e88b3c3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:58.173476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:58.231175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85797624-d94f-44d7-b553-de020e88b3c3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:58.263682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:58.264308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:58.264589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:58.264860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:58.275940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 21 14:48:58.280688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:58.281759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:58.282655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 475/953] 10.222.0.60 () {56 vars in 1096 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:58.288686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:58.291771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:58.383028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c7c4d-8f70-4474-a67e-99299a9e9b84 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:58.417614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:58.418217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:58.418499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:58.418801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:58.419484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:58.419785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:58.421143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:58.428973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 479/954] 10.222.0.60 () {56 vars in 1099 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:58.434852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:58.533698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:58.536623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37bcb1-ab52-47b5-8357-3014e35cc822 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:58.569353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88003928-2357-450d-b5cd-758cb550029b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:58.569971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88003928-2357-450d-b5cd-758cb550029b None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:58.570247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88003928-2357-450d-b5cd-758cb550029b None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:58.570523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88003928-2357-450d-b5cd-758cb550029b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:58.580522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88003928-2357-450d-b5cd-758cb550029b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:58.584431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88003928-2357-450d-b5cd-758cb550029b None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-88003928-2357-450d-b5cd-758cb550029b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:58.596625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:58.597765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:58.599057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:58.599057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:58.599057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:58.599057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 476/955] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:58.603684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:58.668276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:58.701323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7de3f-af01-4a44-a0a5-8da7f1e3f4a9 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:58.733539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:58.734135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:58.734409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:58.734700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:58.735329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:58.735625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:58.737027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:58.748770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 480/956] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:58.754048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88003928-2357-450d-b5cd-758cb550029b None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:58.854894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88003928-2357-450d-b5cd-758cb550029b None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:58.888450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:58.889066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:58.889339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:58.889605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:58.898124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:58.899829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 21 14:48:58.910704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:58.911755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:48:58.912785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 477/957] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:58.917920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.017777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20cf0b4-494e-4578-8fcf-73689af4e042 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:59.043923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:59.052205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:59.052845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:59.053124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:59.053392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:59.054103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:59.054483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:59.055924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:59.073881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 481/958] 10.222.0.60 () {56 vars in 1143 bytes} [Fri Oct 21 14:48:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:59.079747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.095495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:59.180409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81164a8-6408-46ff-91e4-55765312f0cd None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:59.213771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:59.214367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:59.214664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:59.214938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:59.279451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:59.280828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:59.304594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:59.378608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 478/959] 10.222.0.60 () {58 vars in 1282 bytes} [Fri Oct 21 14:48:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 21 14:48:59.384913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.480896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d718ad81-66ce-4094-bae4-187fd35c4516 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:59.514787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:59.515402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:59.515674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:59.515939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:59.526790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:59.528211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 21 14:48:59.542617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:59.543814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 21 14:48:59.544872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 21 14:48:59.544872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 21 14:48:59.544872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:59.544872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 482/960] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:48:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:59.549383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.649228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5331ea95-cacd-44c4-9c26-159189885dab None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:59.662398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:59.683549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:59.684145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:59.684497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:59.684776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:59.685405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:48:59.685704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:59.687079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:59.698837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 479/961] 10.222.0.60 () {56 vars in 1109 bytes} [Fri Oct 21 14:48:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:48:59.704508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.739244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:48:59.803113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b081b-34ea-45e2-956c-1e62a89a957f None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:48:59.837052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:48:59.837736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:48:59.838012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:48:59.838285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:48:59.845133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:48:59.846620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 21 14:48:59.858189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application with Lock( Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:48:59.859266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: ERROR keystone.server.flask.application Oct 21 14:48:59.860210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 483/962] 10.222.0.60 () {56 vars in 1111 bytes} [Fri Oct 21 14:48:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:48:59.865874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:48:59.966543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c16ee-796f-4c14-ac68-e3be8cbb5eea None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:00.002298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:00.002939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:00.003218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:00.003489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:00.004172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:00.004479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:00.006039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f6b77-edc2-407d-a201-e5475348bfb1 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:00.018044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 480/963] 10.222.0.60 () {56 vars in 1148 bytes} [Fri Oct 21 14:48:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:00.029377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:00.116985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:00.122624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e8077f-afbf-43f8-8937-00ade7fcf726 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:00.154937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:00.155533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:00.155815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:00.156087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:00.156800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] RBAC: Adding query filter params (role.id=55a557362b9542629b4209da47492ebf, scope.project.id=d59dd3ce7504494bb481758b4195d883, user.id=e05f107798a24fa9a0e65918076f2cc7) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:00.157106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:00.158530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54501e5-6076-46cd-bc5e-51d9fde080c2 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:00.164847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 484/964] 10.222.0.60 () {56 vars in 1374 bytes} [Fri Oct 21 14:49:00 2022] GET /identity/v3/role_assignments?user.id=e05f107798a24fa9a0e65918076f2cc7&scope.project.id=d59dd3ce7504494bb481758b4195d883&role.id=55a557362b9542629b4209da47492ebf => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:02.320638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc73d1-5eb4-4cac-88cc-3262b4c0e3c4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:02.321163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc73d1-5eb4-4cac-88cc-3262b4c0e3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:02.321163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc73d1-5eb4-4cac-88cc-3262b4c0e3c4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:02.322522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 481/965] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:49:02.328052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:02.328488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:02.328601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:02.346654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:49:02.347854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:49:02.415817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 485/966] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:49:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:49:02.684976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc73d1-5eb4-4cac-88cc-3262b4c0e3c4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:02.716457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:02.786098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc73d1-5eb4-4cac-88cc-3262b4c0e3c4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:02.823644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:02.824274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:02.824549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:02.824817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:02.825402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:02.826788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:49:02.836146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:49:02.837420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:02.838309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 482/967] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:49:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:49:02.843337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:02.931311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:02.940566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2c5f-36e4-4d7c-8256-0537f52f0c11 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:02.974710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:02.975317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:02.975592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:02.975974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:02.976596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:02.976952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:02.978328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279e7fc5-9bcd-41a3-a53f-71c0ea7ebc4e None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:02.983226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 486/968] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Oct 21 14:49:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:02.988964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:03.083114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:03.089138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22579764-8db7-4fdc-b051-2c9c6324e7af None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:03.122935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:03.123535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:03.123811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:03.124081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] PATH_INFO: `/v3/services` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:03.124719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] RBAC: Adding query filter params (type=placement) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:03.125005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:03.126364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea71cac-af6d-4a92-ad3b-eb03c928d2cf None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:03.130698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 483/969] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Oct 21 14:49:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:05.285585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6f269-8045-478f-b9a9-a1d507b05090 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:05.286086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6f269-8045-478f-b9a9-a1d507b05090 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:05.286086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb6f269-8045-478f-b9a9-a1d507b05090 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:05.287539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 487/970] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:49:05.293522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f1f7-134a-4933-8e33-7013a5fcb767 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:05.293953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f1f7-134a-4933-8e33-7013a5fcb767 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:05.294079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b88f1f7-134a-4933-8e33-7013a5fcb767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:05.312234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8b88f1f7-134a-4933-8e33-7013a5fcb767 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:49:05.313425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8b88f1f7-134a-4933-8e33-7013a5fcb767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:49:05.380730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 484/971] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:49:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:49:05.653221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6f269-8045-478f-b9a9-a1d507b05090 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:05.751933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb6f269-8045-478f-b9a9-a1d507b05090 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:05.785099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:05.785690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:05.785965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:05.786231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:05.787016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:05.788360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ca1e0b-00b2-40c9-bb60-642d0113a8e5 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:05.990581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 488/972] 10.222.0.60 () {60 vars in 1141 bytes} [Fri Oct 21 14:49:05 2022] POST /identity/v3/services => generated 246 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:49:08.152787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f017-a5bd-4aee-ba36-3efacd8f971a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:08.153335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f017-a5bd-4aee-ba36-3efacd8f971a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:08.153440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9f017-a5bd-4aee-ba36-3efacd8f971a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:08.154738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 485/973] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:49:08.160221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:08.160656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:08.160749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:08.179419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:49:08.180612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:49:08.189608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:08.252764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 489/974] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:49:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:49:08.526159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9f017-a5bd-4aee-ba36-3efacd8f971a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:08.616761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:08.626919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9f017-a5bd-4aee-ba36-3efacd8f971a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:08.660602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:08.661317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:08.661924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:08.661924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:08.662512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:08.663899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:49:08.672427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:49:08.673588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:08.674882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 486/975] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:49:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:49:08.680270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:08.778568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a50d66-b895-4414-afa0-32ae6eed962e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:08.812348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:08.812942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:08.813217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:08.813485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:08.814097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:08.814496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:08.815987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:08.821570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 490/976] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Oct 21 14:49:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:08.827714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:08.927692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330f58ad-42bf-43e9-86f0-2da265b0b4d8 None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:08.961951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:08.962593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:08.962927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:08.963220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:08.963938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] RBAC: Adding query filter params (interface=public, service_id=17937e1567bc4f89b8efdcea9275d3a9, region_id=RegionOne) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:08.964247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:08.965633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea43c83-ec93-48cb-82ed-987a11d6af43 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:08.970737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 487/977] 10.222.0.60 () {56 vars in 1251 bytes} [Fri Oct 21 14:49:08 2022] GET /identity/v3/endpoints?service_id=17937e1567bc4f89b8efdcea9275d3a9&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:08.977431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:08.981380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:09.076473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ee773-91de-4741-8995-c65f79a23939 None admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:09.109331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:09.109925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:09.110203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:09.110466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:09.111082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:09.112424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4e317-2667-46a4-b078-cb5d73a42867 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:09.119861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 491/978] 10.222.0.60 () {56 vars in 1087 bytes} [Fri Oct 21 14:49:08 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:49:11.266528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b4c0-09c5-4c48-bbca-c619b404f080 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:11.267092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b4c0-09c5-4c48-bbca-c619b404f080 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:11.267092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8b4c0-09c5-4c48-bbca-c619b404f080 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:11.268394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 488/979] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:49:11.274018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:11.276646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:11.276646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:11.292946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:49:11.294147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:49:11.355503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:11.361004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 492/980] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:49:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 21 14:49:11.634030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b4c0-09c5-4c48-bbca-c619b404f080 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:11.654947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:11.742911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8b4c0-09c5-4c48-bbca-c619b404f080 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:11.779739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:11.780372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:11.780648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:11.780913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] PATH_INFO: `/v3/services/placement` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:11.781489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:11.782951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 21 14:49:11.791755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 21 14:49:11.796571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.get_or_create( Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application with Lock( Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self._enter() Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application return self.creator() Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application created_value = creator( Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: ERROR keystone.server.flask.application Oct 21 14:49:11.797417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 489/981] 10.222.0.60 () {56 vars in 1117 bytes} [Fri Oct 21 14:49:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 21 14:49:11.799112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:11.902130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113524f-957b-4d88-934f-754fe4d25a3e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:11.937254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:11.937860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:11.938139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:11.938411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] PATH_INFO: `/v3/services` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:11.939063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 14:49:11.939365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:11.940748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1c133-538c-4e8a-8dd3-22b34b5a5081 None admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:11.946489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 493/982] 10.222.0.60 () {56 vars in 1116 bytes} [Fri Oct 21 14:49:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 14:49:11.952389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:49:12.035126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:12.053775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adcfbf7-7731-48cc-be8d-ae1983373aed None admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:49:12.087459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:49:12.089030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:49:12.089307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:49:12.089580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:49:12.090290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:49:12.091672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc87980c-3053-4d7d-bdc5-d6b3f57b0359 None admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:49:12.166772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 490/983] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 14:49:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 14:49:21.942776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=69 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:49:22.161140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:52.282954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3eb5-c48f-45d2-aad7-142c289f1a99 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:52.284347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3eb5-c48f-45d2-aad7-142c289f1a99 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:52.284993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ba3eb5-c48f-45d2-aad7-142c289f1a99 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:52.292998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b0e5f-40c3-4abe-8fb7-316977390ebd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:52.293421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b0e5f-40c3-4abe-8fb7-316977390ebd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:52.293542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b0e5f-40c3-4abe-8fb7-316977390ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:52.297662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 494/984] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:55:52 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:55:52.403473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ce7b0e5f-40c3-4abe-8fb7-316977390ebd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:55:52.405552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ce7b0e5f-40c3-4abe-8fb7-316977390ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:52.522812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 491/985] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:55:52.822599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:52.823013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:52.823540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:52.824996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 495/986] 10.222.0.60 () {54 vars in 964 bytes} [Fri Oct 21 14:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:55:52.831830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f9d5-3ccd-4ca4-a8d2-bc6ad48074a4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:52.832404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f9d5-3ccd-4ca4-a8d2-bc6ad48074a4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:52.832745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02f9d5-3ccd-4ca4-a8d2-bc6ad48074a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:52.919218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ac02f9d5-3ccd-4ca4-a8d2-bc6ad48074a4 None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:55:52.920995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ac02f9d5-3ccd-4ca4-a8d2-bc6ad48074a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:52.943166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:53.008960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 492/987] 10.222.0.60 () {58 vars in 1061 bytes} [Fri Oct 21 14:55:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:55:53.018123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:55:53.019968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:53.140322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ef869-0a66-452e-acc3-d6c8b149f279 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:55:53.222598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:55:53.223230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:53.223503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:53.223773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:53.225357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:53.284307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:53.341035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:55:53.343069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4515e100-2a30-4ea2-896c-54a14c67a224 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:55:53.363113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 496/988] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:55:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:55:55.866362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272960-da7d-43cb-b0ee-ba3dc8058e7f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:55.866945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272960-da7d-43cb-b0ee-ba3dc8058e7f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:55.866945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84272960-da7d-43cb-b0ee-ba3dc8058e7f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:55.868287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 493/989] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:55:55.874145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4483aa-2d14-4602-b6e3-887f9867c072 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:55.874568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4483aa-2d14-4602-b6e3-887f9867c072 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:55.874696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4483aa-2d14-4602-b6e3-887f9867c072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:55.956699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2d4483aa-2d14-4602-b6e3-887f9867c072 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:55:55.957901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2d4483aa-2d14-4602-b6e3-887f9867c072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:56.053789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 497/990] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:55:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:55:56.351304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:56.351591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:56.351869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:56.353245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 494/991] 10.222.0.60 () {54 vars in 964 bytes} [Fri Oct 21 14:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:55:56.359906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935964fb-bf0e-452d-833d-268e4a3a4076 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:56.360194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935964fb-bf0e-452d-833d-268e4a3a4076 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:56.360469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935964fb-bf0e-452d-833d-268e4a3a4076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:56.372112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:56.435808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-935964fb-bf0e-452d-833d-268e4a3a4076 None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:55:56.437013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-935964fb-bf0e-452d-833d-268e4a3a4076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:56.510516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 498/992] 10.222.0.60 () {58 vars in 1061 bytes} [Fri Oct 21 14:55:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:55:56.520241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:55:56.521740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:56.622990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629a53d0-3bec-4294-bd65-77df605e6c3b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:55:56.709467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:56.711524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:55:56.712118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:56.712392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:56.712661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:56.714218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:56.822312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:55:56.823767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826162e-55ca-4076-b4c2-ca644cce4602 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:55:56.842842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 495/993] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:55:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:55:59.722726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:59.723260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:59.723260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:59.724580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 499/994] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:55:59.729702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd30f-3d52-4f92-918f-9d8dcc096c6b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:55:59.730150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd30f-3d52-4f92-918f-9d8dcc096c6b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:55:59.730249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dd30f-3d52-4f92-918f-9d8dcc096c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:55:59.796104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-367dd30f-3d52-4f92-918f-9d8dcc096c6b None None] MFA Rules not processed for user `ee30fc223ad243bda597e1528974c751`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:55:59.797317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-367dd30f-3d52-4f92-918f-9d8dcc096c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:55:59.849080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:55:59.893330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 496/995] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:55:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:00.181377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:00.183007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:00.287505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79186fb7-feb1-4007-85a7-f101f9854a9f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:00.290258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:00.361631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:00.362229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:00.362503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:00.362798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:00.364357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:00.473568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:00.474932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810a65e-0a6f-432e-955b-c36d756af7e2 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:00.492743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 500/996] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:03.622551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:03.623142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:03.623142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:03.624505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 497/997] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:03.629907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80f1b1-c91f-4333-a922-92be5eab0cd7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:03.630329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80f1b1-c91f-4333-a922-92be5eab0cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:03.630453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80f1b1-c91f-4333-a922-92be5eab0cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:03.672163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:03.696839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ee80f1b1-c91f-4333-a922-92be5eab0cd7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:03.698038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ee80f1b1-c91f-4333-a922-92be5eab0cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:03.792863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 501/998] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:56:04.080246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:04.081772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:04.187211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b0e43-7d3e-473b-815b-928a2db0040c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:04.260488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:04.261013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:04.261310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:04.261576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:04.263181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:04.270457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:04.371519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:04.372996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e747de-eefe-45cd-aace-f40010164757 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:04.390386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 498/999] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:07.584645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:07.585194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:07.585194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:07.586502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 502/1000] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:07.592231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b98f-3a44-47f7-92c6-94c61f14b590 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:07.592661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b98f-3a44-47f7-92c6-94c61f14b590 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:07.592834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed8b98f-3a44-47f7-92c6-94c61f14b590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:07.670166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-aed8b98f-3a44-47f7-92c6-94c61f14b590 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:07.671486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-aed8b98f-3a44-47f7-92c6-94c61f14b590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:07.733690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:07.774175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 499/1001] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:08.061601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:08.063219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:08.167710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f4e7d-ac77-48a3-9ad3-3222e5b8d1a1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:08.181582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:08.242435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:08.243117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:08.243472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:08.243748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:08.245325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:08.359776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:08.361123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642fb1-19db-4364-a84f-a0c537beafcd service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:08.380896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 503/1002] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:10.584972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:10.585558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:10.585558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:10.586917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 500/1003] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:10.592207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd4722-55b0-4b03-9315-5100ee7c3459 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:10.592633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd4722-55b0-4b03-9315-5100ee7c3459 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:10.592745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fd4722-55b0-4b03-9315-5100ee7c3459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:10.640983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:10.659087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a8fd4722-55b0-4b03-9315-5100ee7c3459 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:10.660381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a8fd4722-55b0-4b03-9315-5100ee7c3459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:10.767139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 504/1004] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:11.050253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:11.051812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:11.159065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5172238-7ddb-4c6b-8d1b-85ac7002f5c2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:11.234951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:11.235553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:11.235826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:11.236095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:11.237676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:11.254623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:11.350480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:11.351924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccadbce0-5b81-490c-a34c-b91e9dbc57cc service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:11.371232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 501/1005] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:13.599311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:13.599942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:13.600227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:13.601571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 505/1006] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:13.606730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e495-739b-4840-8040-c0a374a32f56 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:13.607131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e495-739b-4840-8040-c0a374a32f56 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:13.607346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db4e495-739b-4840-8040-c0a374a32f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:13.679239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2db4e495-739b-4840-8040-c0a374a32f56 None None] MFA Rules not processed for user `ee30fc223ad243bda597e1528974c751`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:13.680454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2db4e495-739b-4840-8040-c0a374a32f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:13.750432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:13.787450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 502/1007] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:13 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:14.078301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:14.085695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:14.201963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1f278b-e2ba-4be1-bbea-ab7a051e13e2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:14.225333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:14.281350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:14.281890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:14.282174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:14.282743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:14.284362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:14.394895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:14.396321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15611e-9d09-4e07-b575-ac7acbdddec3 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:14.415894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 506/1008] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:14 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 14:56:17.040271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:17.040795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:17.040795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:17.042140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 503/1009] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:17.047607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d52c-c23f-411a-8079-5459540cd5bf None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:17.048037 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d52c-c23f-411a-8079-5459540cd5bf None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:17.048157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3039d52c-c23f-411a-8079-5459540cd5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:17.113253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:17.115893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3039d52c-c23f-411a-8079-5459540cd5bf None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:17.117092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3039d52c-c23f-411a-8079-5459540cd5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:17.214556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 507/1010] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:17.495064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:17.496596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:17.599562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d7638-3906-4065-83d1-6db017b1d86e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:17.673550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:17.674153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:17.674436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:17.674731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:17.676304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:17.703276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:17.783553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:17.784909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66f92a3-c32e-4943-a837-19b1b5fecae5 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:17.803356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 504/1011] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 14:56:19.997088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:19.997565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:19.997702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:19.999033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 508/1012] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:20.004363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc23e-2617-4b59-8976-872fbe96f6f9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:20.004761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc23e-2617-4b59-8976-872fbe96f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:20.004914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecfc23e-2617-4b59-8976-872fbe96f6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:20.071955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3ecfc23e-2617-4b59-8976-872fbe96f6f9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:20.077299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3ecfc23e-2617-4b59-8976-872fbe96f6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:20.149372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:20.170877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 505/1013] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:20.453385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:20.454995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:20.558648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a503159-26c6-45c4-858d-3da5e60fad3d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:20.587051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:20.632461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:20.633063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:20.633338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:20.633605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:20.635207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:20.743814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:20.745159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0aa5e3-9037-4103-b0ba-17b61d665689 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:20.763439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 509/1014] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:23.452441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:23.452995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:23.452995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:23.454296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 506/1015] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:23.459373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb17af-ab6b-4b55-9592-cb4c41ab708b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:23.459816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb17af-ab6b-4b55-9592-cb4c41ab708b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:23.459932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb17af-ab6b-4b55-9592-cb4c41ab708b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:23.526655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2ccb17af-ab6b-4b55-9592-cb4c41ab708b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:23.527856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2ccb17af-ab6b-4b55-9592-cb4c41ab708b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:23.537255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:23.626260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 510/1016] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:23.913050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:23.914575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:24.020519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd552aaa-c2a5-419b-be50-773efe763c60 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:24.096485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:24.097088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:24.097365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:24.097635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:24.099252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:24.134458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:24.209294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:24.210750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1790d202-3cbe-4775-a5a3-8f97caa73847 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:24.229016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 507/1017] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:28.640066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:28.640564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:28.640649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:28.641983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 511/1018] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:28.647534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34acbd4-20e9-4116-8665-b540e19ba806 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:28.647977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34acbd4-20e9-4116-8665-b540e19ba806 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:28.648127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34acbd4-20e9-4116-8665-b540e19ba806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:28.715738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c34acbd4-20e9-4116-8665-b540e19ba806 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:28.716982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c34acbd4-20e9-4116-8665-b540e19ba806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:28.803486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:28.816003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 508/1019] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:29.098957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:29.100482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:29.207481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a37e70-c7b8-4da0-96ab-f04f4f009507 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:29.246000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:29.285053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:29.285666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:29.285947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:29.286220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:29.287861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:29.398556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:29.400032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734ac57d-cc33-490f-925b-b69769b17913 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:29.419257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 512/1020] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:32.986982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:32.987475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:32.987568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:32.988873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 509/1021] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:32.994022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1879e-03c6-4d49-8b4f-47bdb2e74594 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:32.994433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1879e-03c6-4d49-8b4f-47bdb2e74594 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:32.994561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc1879e-03c6-4d49-8b4f-47bdb2e74594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:33.062329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7dc1879e-03c6-4d49-8b4f-47bdb2e74594 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:33.063575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7dc1879e-03c6-4d49-8b4f-47bdb2e74594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:33.081983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:33.164605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 513/1022] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:33.449775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:33.452004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:33.557123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350349fb-6c2a-49bc-b2cd-c80f2dd85f3e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:33.643701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:33.654446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:33.654446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:33.654446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:33.654446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:33.713994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:33.798835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:33.805350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e7d15-5f51-4733-ba10-fb7f92a3ee34 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:33.821466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 510/1023] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:38.400188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:38.400741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:38.400741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:38.402074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 514/1024] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:38.407512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbff6f-360f-4e74-9f91-2851e6f3948b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:38.408202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbff6f-360f-4e74-9f91-2851e6f3948b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:38.408320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fbff6f-360f-4e74-9f91-2851e6f3948b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:38.478153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f8fbff6f-360f-4e74-9f91-2851e6f3948b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:38.479436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f8fbff6f-360f-4e74-9f91-2851e6f3948b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:38.579237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:38.581417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 511/1025] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:38.863951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:38.865464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:38.969002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eb9361-4ed3-4991-816a-e0ffd0216aca None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:39.016879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:39.042617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:39.043251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:39.043528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:39.043798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:39.046346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:39.162733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:39.164197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c465c-77d9-4f26-8766-64a01cd844a6 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:39.182365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 515/1026] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:44.211790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146b57b-a4c6-4a06-bc46-b965e216b720 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:44.212247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146b57b-a4c6-4a06-bc46-b965e216b720 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:44.212354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6146b57b-a4c6-4a06-bc46-b965e216b720 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:44.213733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 512/1027] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 14:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:44.230056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e075-e85e-42ef-922a-76154e85a926 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:44.230519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e075-e85e-42ef-922a-76154e85a926 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:44.230687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f3e075-e85e-42ef-922a-76154e85a926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:44.343577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a0f3e075-e85e-42ef-922a-76154e85a926 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:44.345241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a0f3e075-e85e-42ef-922a-76154e85a926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:44.382707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:44.470276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 516/1028] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 14:56:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:48.663490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b41a9-f0ff-49f6-8881-8c082a180f17 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:48.664048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b41a9-f0ff-49f6-8881-8c082a180f17 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:48.664048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151b41a9-f0ff-49f6-8881-8c082a180f17 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:48.665608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 513/1029] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 14:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:48.670528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d75a0-1223-4670-8bed-def559225be5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:48.671041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d75a0-1223-4670-8bed-def559225be5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:48.671333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d75a0-1223-4670-8bed-def559225be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:48.754312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c37d75a0-1223-4670-8bed-def559225be5 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:48.755749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c37d75a0-1223-4670-8bed-def559225be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:48.840557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:48.887564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 517/1030] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 14:56:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:50.199705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9525977-c66a-40a9-8a78-5e16981f6d20 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:50.200265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9525977-c66a-40a9-8a78-5e16981f6d20 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:50.200265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9525977-c66a-40a9-8a78-5e16981f6d20 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:50.201763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 514/1031] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 14:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:50.207060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68af85e-21c7-4e01-9771-f084dd3aa7c4 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:50.207906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68af85e-21c7-4e01-9771-f084dd3aa7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:50.208234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68af85e-21c7-4e01-9771-f084dd3aa7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:50.288373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a68af85e-21c7-4e01-9771-f084dd3aa7c4 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:50.289585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a68af85e-21c7-4e01-9771-f084dd3aa7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:50.399787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 518/1032] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 14:56:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:53.968374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056516d6-92ad-4692-a4dd-8745b629cb9a None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:53.968925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056516d6-92ad-4692-a4dd-8745b629cb9a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:53.968925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056516d6-92ad-4692-a4dd-8745b629cb9a None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:53.977744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e815ff-8824-4761-82ef-4f5cbda952ca None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:53.978263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e815ff-8824-4761-82ef-4f5cbda952ca None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:53.978356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e815ff-8824-4761-82ef-4f5cbda952ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:53.986031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 515/1033] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:53 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:54.029618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:54.071347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d9e815ff-8824-4761-82ef-4f5cbda952ca None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:54.073129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d9e815ff-8824-4761-82ef-4f5cbda952ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:54.190241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 519/1034] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:54.489982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:54.490474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:54.490929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:54.492723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 516/1035] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 14:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:54.506597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62937c9-48c4-4441-9046-0e72d03e6fb5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:54.507016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62937c9-48c4-4441-9046-0e72d03e6fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:54.507419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62937c9-48c4-4441-9046-0e72d03e6fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:54.596919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f62937c9-48c4-4441-9046-0e72d03e6fb5 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:54.598396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f62937c9-48c4-4441-9046-0e72d03e6fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:54.654216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:54.729599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 520/1036] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 14:56:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:54.737329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:54.739380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:54.900064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33689047-9b9a-4424-aae2-ea720b0bdab2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:54.983732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:55.026736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:55.027466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:55.027827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:55.028311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:55.030280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:55.136286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:55.138116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213140a-98f0-46ce-ae67-1e9832efbbf3 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:55.157534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 517/1037] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:56:57.577121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:57.577647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:57.577647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:57.579021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 521/1038] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:56:57.585774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61674-e17c-46e1-94b8-05d8cc90d253 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:57.586273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61674-e17c-46e1-94b8-05d8cc90d253 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:57.586273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f61674-e17c-46e1-94b8-05d8cc90d253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:57.656268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a5f61674-e17c-46e1-94b8-05d8cc90d253 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:56:57.657449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a5f61674-e17c-46e1-94b8-05d8cc90d253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:57.702947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:57.765250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 518/1039] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:56:58.062019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:56:58.063566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:58.249730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f27747-6513-409d-9756-81aa37c03b35 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:56:58.260991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:56:58.383848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:56:58.384467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:56:58.384754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:56:58.385025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:56:58.386681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:56:58.480406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:56:58.483386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42e51ad-db4d-44c6-a546-f064555b8bfc service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:56:58.501537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 522/1040] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:01.095415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fec-cf0c-4a4a-962a-ad3fcd950ddf None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:01.096382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fec-cf0c-4a4a-962a-ad3fcd950ddf None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:01.096783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a447fec-cf0c-4a4a-962a-ad3fcd950ddf None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:01.102389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 519/1041] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:01.108594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bd879-b444-4235-b424-1682aeee8758 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:01.109095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bd879-b444-4235-b424-1682aeee8758 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:01.109192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761bd879-b444-4235-b424-1682aeee8758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:01.125978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:01.181204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-761bd879-b444-4235-b424-1682aeee8758 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:01.186448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-761bd879-b444-4235-b424-1682aeee8758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:01.294187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 523/1042] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:01.587747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:01.588347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:01.588844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:01.591335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 520/1043] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 14:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:01.596442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4377e3c-5bae-45d8-b7dc-d21cb8b418a6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:01.596836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4377e3c-5bae-45d8-b7dc-d21cb8b418a6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:01.597209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4377e3c-5bae-45d8-b7dc-d21cb8b418a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:01.678888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d4377e3c-5bae-45d8-b7dc-d21cb8b418a6 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:01.680472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d4377e3c-5bae-45d8-b7dc-d21cb8b418a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:01.719060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:01.805189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 524/1044] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 14:57:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:01.813150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:01.815996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:01.971482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c66fa9-ff27-4e68-aab0-7a5a3260b110 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:01.996884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:02.102103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:02.103061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:02.103344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:02.103878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:02.105625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:02.197838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68762-ebf5-4533-b566-bf678ee656de None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:02.198414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68762-ebf5-4533-b566-bf678ee656de None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:02.198414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d68762-ebf5-4533-b566-bf678ee656de None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:02.200437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 525/1045] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:57:02.202591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:02.204314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603092b-410a-428a-a46f-46aef15a4fb6 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:02.204769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806607-e1ce-4ede-b1fc-71ab7da85103 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:02.205048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806607-e1ce-4ede-b1fc-71ab7da85103 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:02.205317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79806607-e1ce-4ede-b1fc-71ab7da85103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:02.225103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 521/1046] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:02.276107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-79806607-e1ce-4ede-b1fc-71ab7da85103 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:02.277391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-79806607-e1ce-4ede-b1fc-71ab7da85103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:02.385363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:02.393508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 526/1047] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:57:02.741714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:02.741997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:02.742266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:02.743770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 522/1048] 10.222.0.60 () {54 vars in 963 bytes} [Fri Oct 21 14:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:02.748646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b26c8-4586-4050-8258-d2a03dd1e0df None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:02.749037 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b26c8-4586-4050-8258-d2a03dd1e0df None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:02.749414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b26c8-4586-4050-8258-d2a03dd1e0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:02.832998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e43b26c8-4586-4050-8258-d2a03dd1e0df None None] MFA Rules not processed for user `33932a27fdc0413fac63db9898ef8475`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:02.834393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e43b26c8-4586-4050-8258-d2a03dd1e0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:02.915924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 527/1049] 10.222.0.60 () {58 vars in 1060 bytes} [Fri Oct 21 14:57:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:02.923698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:02.925193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:02.959456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:03.041837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf5765-ab26-41f6-89da-37673d161807 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:03.127791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:03.128987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:03.128987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:03.128987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:03.130957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:03.214937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:03.253446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:03.259016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5352575e-dd28-4daa-84ba-d0e5780d9319 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:03.273682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 523/1050] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:04.645223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:04.645808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:04.645808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:04.647373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 528/1051] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:04.652566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3c09d-2818-4b9b-87fd-da7e78e93699 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:04.653523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3c09d-2818-4b9b-87fd-da7e78e93699 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:04.653804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3c09d-2818-4b9b-87fd-da7e78e93699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:04.724184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9bb3c09d-2818-4b9b-87fd-da7e78e93699 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:04.725384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9bb3c09d-2818-4b9b-87fd-da7e78e93699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:04.822833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 524/1052] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:05.121273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:05.123218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:05.211714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:05.276342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63698cd-6fa4-4dd7-953c-74a45ce4aba3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:05.399926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:05.400545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:05.400856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:05.401127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:05.402743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:05.458348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:05.501696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:05.503384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61a0e48-5456-4655-a696-3608cfa9bf03 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:05.522405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 529/1053] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:07.874854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:07.875405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:07.875405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:07.878935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 525/1054] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:07.881861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2956c-efe1-4052-8d69-8fb3ee81641b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:07.882243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2956c-efe1-4052-8d69-8fb3ee81641b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:07.882413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2956c-efe1-4052-8d69-8fb3ee81641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:07.955417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-09b2956c-efe1-4052-8d69-8fb3ee81641b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:07.956972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-09b2956c-efe1-4052-8d69-8fb3ee81641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:08.058422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 530/1055] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:08.348107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:08.349758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:08.389706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:08.504088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed706a4d-7fc4-4748-8c7d-9769e0b897db None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:08.628080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:08.633993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:08.634601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:08.634913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:08.635176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:08.636776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:08.736236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:08.737800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2afd4a3-8448-4763-98b1-35f5a8c813fc service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:08.756455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 526/1056] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:11.125652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:11.126286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:11.126286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:11.127554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 531/1057] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:11.132869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f7ed6-cb94-421c-914c-b093594b445b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:11.133298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f7ed6-cb94-421c-914c-b093594b445b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:11.133415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8f7ed6-cb94-421c-914c-b093594b445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:11.204784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1d8f7ed6-cb94-421c-914c-b093594b445b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:11.206015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1d8f7ed6-cb94-421c-914c-b093594b445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:11.272205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:11.306124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 527/1058] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:11.598318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:11.599904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:11.629458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:11.752513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2d4c4-6c28-4303-af37-08c3d91e855a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:11.851783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:11.867753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:11.868369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:11.868649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:11.868924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:11.870555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:11.989253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:11.991405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29012c12-32b9-4412-addf-ae5ddb560afd service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:12.016835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 532/1059] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:14.686578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:14.687316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:14.687316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:14.688496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 528/1060] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:14.694169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceac24-2f68-413f-8858-40e79724e5e6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:14.694809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceac24-2f68-413f-8858-40e79724e5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:14.694809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceac24-2f68-413f-8858-40e79724e5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:14.762284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-91ceac24-2f68-413f-8858-40e79724e5e6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:14.763687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-91ceac24-2f68-413f-8858-40e79724e5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:14.826080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:14.871085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 533/1061] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:15.166657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:15.168267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:15.318267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd321903-5297-475f-816e-b1a792ab07f4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:15.372278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:15.441377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:15.442070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:15.443047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:15.443338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:15.444999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:15.537857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:15.539634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0894122a-a496-45d9-a1e3-4c1bb0079e3d service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:15.558198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 529/1062] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:17.724681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:17.725212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:17.725328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:17.726603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 534/1063] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:17.732116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78251-8415-45de-bccc-16b1b6997206 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:17.732548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78251-8415-45de-bccc-16b1b6997206 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:17.732714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c78251-8415-45de-bccc-16b1b6997206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:17.800383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-85c78251-8415-45de-bccc-16b1b6997206 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:17.801593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-85c78251-8415-45de-bccc-16b1b6997206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:17.810051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:17.900867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 530/1064] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:18.190544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:18.192110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:18.341202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0895f55-5caf-43aa-a866-921b529b5002 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:18.389316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:18.458661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:18.459273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:18.459549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:18.459819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:18.461395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:18.553116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:18.554721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97c0073-1882-418d-a16c-2a11ba995ac1 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:18.572677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 535/1065] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:18.767872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f527-c151-41f8-a90d-b1d74ae29844 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:18.768412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f527-c151-41f8-a90d-b1d74ae29844 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:18.768412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b061f527-c151-41f8-a90d-b1d74ae29844 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:18.769881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 531/1066] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:18.775708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168064-1299-4abd-9f34-4242e57a5a50 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:18.776179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168064-1299-4abd-9f34-4242e57a5a50 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:18.781814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd168064-1299-4abd-9f34-4242e57a5a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:18.851415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:18.854747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dd168064-1299-4abd-9f34-4242e57a5a50 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:18.855993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dd168064-1299-4abd-9f34-4242e57a5a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:18.965939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 536/1067] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:19.270225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:19.270609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:19.270918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:19.272278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 532/1068] 10.222.0.60 () {54 vars in 962 bytes} [Fri Oct 21 14:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:19.277508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee8f-a7d1-4473-bfbb-680cce00913c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:19.277796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee8f-a7d1-4473-bfbb-680cce00913c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:19.278073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bdee8f-a7d1-4473-bfbb-680cce00913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:19.357442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-48bdee8f-a7d1-4473-bfbb-680cce00913c None None] MFA Rules not processed for user `fccd7b631a914ab684da6d642501cb64`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:19.359290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-48bdee8f-a7d1-4473-bfbb-680cce00913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:19.428848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:19.434508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 537/1069] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Oct 21 14:57:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:57:19.443552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:19.445182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:19.550410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad20a66-e0bd-4d9a-a2d0-c3fc2caa8ae5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:19.588672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:19.626460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:19.627121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:19.627397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:19.627670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:19.629382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:19.741095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:19.742662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59915b9c-020b-4546-984b-d11b0246c3a2 service glance] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:19.762264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 533/1070] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 14:57:20.672300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25e83-8738-4435-a2e2-df8676747308 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:20.672796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25e83-8738-4435-a2e2-df8676747308 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:20.673028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa25e83-8738-4435-a2e2-df8676747308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:20.743111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-9fa25e83-8738-4435-a2e2-df8676747308 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:20.744357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9fa25e83-8738-4435-a2e2-df8676747308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:20.837730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 538/1071] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:20.886012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e210-0abf-4117-bdd1-7c18942f47e3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:20.886528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e210-0abf-4117-bdd1-7c18942f47e3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:20.886528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3e210-0abf-4117-bdd1-7c18942f47e3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:20.887976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 534/1072] 10.222.0.60 () {54 vars in 943 bytes} [Fri Oct 21 14:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:20.894879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:20.895170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:20.895449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:20.932176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfc0a7-dae8-4d2d-9053-4293b50c11c4 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:20.932472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfc0a7-dae8-4d2d-9053-4293b50c11c4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:20.932731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfc0a7-dae8-4d2d-9053-4293b50c11c4 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:20.934333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 535/1073] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:57:20.940003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:20.940278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:20.940545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:20.980283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] MFA Rules not processed for user `d34f518f5eef401da71b7624c9917d44`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:20.981128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:21.009263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:21.011946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:21.013194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:21.033311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:21.094587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 539/1074] 10.222.0.60 () {58 vars in 1040 bytes} [Fri Oct 21 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:21.104227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:21.114542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf36350a-7a4c-4ae9-b25b-734b2f0a3822 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:21.117356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 536/1075] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:21.219864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:21.220473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:21.220756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:21.221102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:21.222764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:21.257246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:21.310203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:21.311854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0798cb-cad7-49fc-b87c-e764a40eb143 service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:21.319844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 540/1076] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 14:57:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 21 14:57:21.415598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:21.417100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:21.532318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3cd93e-30c1-4d9e-9093-87b94eaa700f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:21.569571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:21.647368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:21.647982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:21.648259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:21.648535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:21.650121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:21.753341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:21.755364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8467487-1bda-4d6b-83b2-f2595f445396 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:21.775675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 537/1077] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:24.143551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:24.144140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:24.144140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:24.145807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 541/1078] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:24.150919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4391-6535-4b7f-96e2-ea098c1fc0d3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:24.151384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4391-6535-4b7f-96e2-ea098c1fc0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:24.151473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e4391-6535-4b7f-96e2-ea098c1fc0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:24.207063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:24.226586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4a1e4391-6535-4b7f-96e2-ea098c1fc0d3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:24.228487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4a1e4391-6535-4b7f-96e2-ea098c1fc0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:24.350768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 538/1079] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:24.639398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:24.641085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:24.812365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f294d3-8320-49ec-aab6-bac9ce5475e2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:24.843469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:24.941009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:24.941624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:24.941901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:24.942172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:24.943890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:25.057197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:25.058945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76338de7-4b05-4910-b47e-0f7bd516b59c service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:25.079434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 542/1080] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 14:57:27.405755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:27.406341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:27.406341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:27.413612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f107ff-9cf7-47bd-a3ff-fa223c4c98c2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:27.414193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f107ff-9cf7-47bd-a3ff-fa223c4c98c2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:27.414193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f107ff-9cf7-47bd-a3ff-fa223c4c98c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:27.419202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 539/1081] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:27 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:27.475544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:27.520349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c1f107ff-9cf7-47bd-a3ff-fa223c4c98c2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:27.523534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c1f107ff-9cf7-47bd-a3ff-fa223c4c98c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:27.678824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 543/1082] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:57:27.962609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:27.964532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:28.115531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:28.157914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643515c-abe0-4845-91a5-d46bd4d3cc53 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:28.313673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:28.314208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:28.314486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:28.314793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:28.316441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:28.433305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:28.434931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bb703-97d7-46a9-a7a0-97145a66ed5f service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:28.456514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 540/1083] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:28.512809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:30.883320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:30.883885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:30.883885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:30.885278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 544/1084] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:57:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:30.891637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19975-2e67-4da5-87ad-4377414dbd37 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:30.892112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19975-2e67-4da5-87ad-4377414dbd37 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:30.892214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf19975-2e67-4da5-87ad-4377414dbd37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:30.967899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0cf19975-2e67-4da5-87ad-4377414dbd37 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:30.969170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0cf19975-2e67-4da5-87ad-4377414dbd37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:31.078430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 541/1085] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:57:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:31.368855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:31.370375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:31.485722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:31.532907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcba552-6d3b-498c-a6cd-f84acf86d003 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:31.658914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:31.659446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:31.659721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:31.659988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:31.661577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:31.751346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:31.768438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:31.770135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71ddfef-22ea-4470-a775-939662156be9 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:31.789616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 545/1086] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:41.076227 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=74 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:41.792150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=13 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:45.645736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529baf1-8cf3-42ce-a4cf-f9dde3e5f6b2 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:45.646321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529baf1-8cf3-42ce-a4cf-f9dde3e5f6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:45.646321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b529baf1-8cf3-42ce-a4cf-f9dde3e5f6b2 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:45.647632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 542/1087] 10.222.0.60 () {54 vars in 918 bytes} [Fri Oct 21 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:45.652894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e2ee-2c39-4c55-acda-15508e962721 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:45.653320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e2ee-2c39-4c55-acda-15508e962721 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:45.653422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3e2ee-2c39-4c55-acda-15508e962721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:45.729001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-eda3e2ee-2c39-4c55-acda-15508e962721 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:45.730195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-eda3e2ee-2c39-4c55-acda-15508e962721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:45.836673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 546/1088] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Oct 21 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:45.851229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00286e-a64b-4431-8dd8-76bbe1d6d3f3 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:45.851510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00286e-a64b-4431-8dd8-76bbe1d6d3f3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:45.851779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df00286e-a64b-4431-8dd8-76bbe1d6d3f3 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:45.853092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 543/1089] 10.222.0.60 () {54 vars in 918 bytes} [Fri Oct 21 14:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:57:45.857636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7410e-e155-4d5d-a588-6cb63660a1e2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:45.857922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7410e-e155-4d5d-a588-6cb63660a1e2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:45.858195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce7410e-e155-4d5d-a588-6cb63660a1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:45.932847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:45.935847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2ce7410e-e155-4d5d-a588-6cb63660a1e2 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:45.937069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2ce7410e-e155-4d5d-a588-6cb63660a1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:46.055901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 547/1090] 10.222.0.60 () {58 vars in 1015 bytes} [Fri Oct 21 14:57:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:57:48.645834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:48.646394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:48.646394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:48.647760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 544/1091] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:48.653219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74071-a93b-4436-aaa8-1345510fd111 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:48.653646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74071-a93b-4436-aaa8-1345510fd111 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:48.653748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c74071-a93b-4436-aaa8-1345510fd111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:48.721294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c1c74071-a93b-4436-aaa8-1345510fd111 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:48.722510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c1c74071-a93b-4436-aaa8-1345510fd111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:48.779727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:48.822750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 548/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:49.117812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:49.119383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:49.266067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d578e5f-5c5d-46b2-b28a-7b1ca40aa663 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:49.339106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:49.378453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:49.379140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:49.379599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:49.380065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:49.383024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:49.472785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:49.474362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778650a5-857c-429b-ac4d-fe11cd31f026 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:49.492670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 545/1093] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:50.632075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:50.632505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:50.632641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:50.633939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 549/1094] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:50.640089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a359-dd24-42f8-b97f-002b4cc34008 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:50.641576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a359-dd24-42f8-b97f-002b4cc34008 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:50.641576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d70a359-dd24-42f8-b97f-002b4cc34008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:50.718692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0d70a359-dd24-42f8-b97f-002b4cc34008 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:50.719850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0d70a359-dd24-42f8-b97f-002b4cc34008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:50.746112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:50.815878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 546/1095] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:50.972243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:50.973786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:51.119656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c2459-46d9-4173-9169-ae127f82c747 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:51.160913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:51.233628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:51.234167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:51.234445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:51.234742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:51.236349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:51.334221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:51.335858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94cdc6-9af6-4aa9-b66f-3b207430b655 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:51.355620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 550/1096] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:52.501802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225084ce-c807-401a-a944-d18b6844400c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:52.502315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225084ce-c807-401a-a944-d18b6844400c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:52.502441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225084ce-c807-401a-a944-d18b6844400c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:52.503872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 547/1097] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:52.510323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb6e6a-603e-4cfc-b7e5-1b9c8ef9496f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:52.510762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb6e6a-603e-4cfc-b7e5-1b9c8ef9496f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:52.510920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb6e6a-603e-4cfc-b7e5-1b9c8ef9496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:52.567607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:52.589201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4bbb6e6a-603e-4cfc-b7e5-1b9c8ef9496f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:52.590399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4bbb6e6a-603e-4cfc-b7e5-1b9c8ef9496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:52.687599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 551/1098] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:52.843428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225084ce-c807-401a-a944-d18b6844400c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:52.845094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-225084ce-c807-401a-a944-d18b6844400c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:52.990552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225084ce-c807-401a-a944-d18b6844400c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:53.003118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:53.103378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:53.103979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:53.104256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:53.104526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:53.106109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:53.195376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:53.196953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd8dce0-3e21-4694-9c96-4d7e7a9f061a service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:53.214883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 548/1099] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:54.330813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:54.331359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:54.331359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:54.332626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 552/1100] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:54.338681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297072-d882-47fb-96d7-27a910283694 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:54.339103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297072-d882-47fb-96d7-27a910283694 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:54.339228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297072-d882-47fb-96d7-27a910283694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:54.360004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:54.418282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-88297072-d882-47fb-96d7-27a910283694 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:54.419530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-88297072-d882-47fb-96d7-27a910283694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:54.516048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 549/1101] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:54.665232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:54.666728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:54.776364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:54.812052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05fcedc-e52b-4539-87cd-4b7755cfbd69 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:54.924835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:54.925438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:54.925716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:54.925985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:54.927586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:55.005415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:55.018865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:55.020432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1804688-f69e-4e53-ad4f-4db725dfc58b service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:55.039602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 553/1102] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:56.154994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:56.155528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:56.155528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:56.157033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 550/1103] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:56.163608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d720-cfcd-4a15-9bf7-528045529fea None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:56.164024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d720-cfcd-4a15-9bf7-528045529fea None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:56.164151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312d720-cfcd-4a15-9bf7-528045529fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:56.239170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7312d720-cfcd-4a15-9bf7-528045529fea None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:56.240368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7312d720-cfcd-4a15-9bf7-528045529fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:56.335594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 554/1104] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:56.485475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:56.487032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:56.571935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:56.633633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8732a-a0a1-4e31-8545-52fa625f3349 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:56.746270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:56.746900 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:56.747179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:56.747452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:56.749045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:56.799314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:56.841352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:56.842889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5e765f-7cd1-4221-83ec-d01feaa76191 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:56.861367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 551/1105] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:57.970441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:57.970998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:57.970998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:57.972255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 555/1106] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:57.978776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383237-ca22-4505-9c56-bcddf8e39468 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:57.979223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383237-ca22-4505-9c56-bcddf8e39468 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:57.980406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29383237-ca22-4505-9c56-bcddf8e39468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:58.055733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-29383237-ca22-4505-9c56-bcddf8e39468 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:58.056917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-29383237-ca22-4505-9c56-bcddf8e39468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:58.152494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 552/1107] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:57:58.306814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:57:58.308241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:58.357387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:58.455056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df457b6-f51e-4954-ba87-d4a0475b5625 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:57:58.566017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:57:58.566554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:58.566863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:58.567134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:58.568726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:58.575517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:58.660980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:57:58.662616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03814b2-b953-4fa9-990a-0cf5e820e4ca service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:57:58.680187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 556/1108] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:57:59.779981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:59.780488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:59.780600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:59.782459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 553/1109] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:57:59.788894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd82ec-3915-4e28-9c5e-27901e3585d4 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:57:59.789332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd82ec-3915-4e28-9c5e-27901e3585d4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:57:59.789428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd82ec-3915-4e28-9c5e-27901e3585d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:57:59.863863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-63cd82ec-3915-4e28-9c5e-27901e3585d4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:57:59.865048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-63cd82ec-3915-4e28-9c5e-27901e3585d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:57:59.931188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:57:59.960008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 557/1110] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:57:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:00.112927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:00.114443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:00.132986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:00.262371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd4254-a140-42f3-b243-03777510f58d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:00.351478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:00.377106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:00.377702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:00.377978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:00.378246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:00.379841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:00.469338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:00.470936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc84fdd-6afd-4e7f-b0c3-f664d114a053 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:00.489075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 554/1111] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:01.596025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:01.596596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:01.596596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:01.597898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 558/1112] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:01.604014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea326e3-cb3a-4f3d-8486-92b121cf5ee3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:01.604436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea326e3-cb3a-4f3d-8486-92b121cf5ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:01.604564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea326e3-cb3a-4f3d-8486-92b121cf5ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:01.680158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cea326e3-cb3a-4f3d-8486-92b121cf5ee3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:01.681375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cea326e3-cb3a-4f3d-8486-92b121cf5ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:01.723931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:01.778415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 555/1113] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:58:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:01.927580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:01.929085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:02.076751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c70ff7-c816-4c06-b534-e83b4658b89d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:02.131523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:02.189628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:02.190168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:02.190447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:02.190748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:02.192343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:02.282121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:02.283728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbd8cc-d9ed-479a-801e-4649feaae403 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:02.301592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 559/1114] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:03.416728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:03.417261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:03.417261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:03.418586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 556/1115] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 14:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:03.425462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e871-378b-40f0-9c21-63a33bfedbcd None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:03.425921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e871-378b-40f0-9c21-63a33bfedbcd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:03.426018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695e871-378b-40f0-9c21-63a33bfedbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:03.502052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4695e871-378b-40f0-9c21-63a33bfedbcd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:03.503289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4695e871-378b-40f0-9c21-63a33bfedbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:03.505276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:03.600942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 560/1116] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 14:58:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:03.750939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:03.752438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:03.899022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f44fe4-116b-4571-bb60-1f52276453e1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:03.926139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:04.015078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:04.016128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:04.016419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:04.016682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:04.018265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:04.107982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:04.109591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a378-a54d-4837-acc0-1106b20bf0b3 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:04.128781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 557/1117] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:06.211671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:06.212261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:06.212261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:06.213529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 561/1118] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:06.219496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca4d8-f11e-4856-8fc4-eac1610cfb8f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:06.219966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca4d8-f11e-4856-8fc4-eac1610cfb8f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:06.220105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ca4d8-f11e-4856-8fc4-eac1610cfb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:06.260835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:06.286320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-631ca4d8-f11e-4856-8fc4-eac1610cfb8f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:06.287557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-631ca4d8-f11e-4856-8fc4-eac1610cfb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:06.383981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 558/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:06.667560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:06.669087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:06.771909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b878f-4742-4e4c-ba04-0f25d5a596a5 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:06.803991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:06.845120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:06.845726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:06.846001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:06.846270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:06.847895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:06.961713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:06.963205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997b6c0f-f090-47d3-b01a-aa42f90ff27e service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:06.981206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 562/1120] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:09.366403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:09.366969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:09.366969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:09.368250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 559/1121] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:09.373629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b83b12-a9d3-44e4-a263-b2ad089f226d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:09.374056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b83b12-a9d3-44e4-a263-b2ad089f226d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:09.374175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b83b12-a9d3-44e4-a263-b2ad089f226d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:09.439821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-22b83b12-a9d3-44e4-a263-b2ad089f226d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:09.441014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-22b83b12-a9d3-44e4-a263-b2ad089f226d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:09.453603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:09.539616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 563/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:09.826674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:09.828979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:09.934249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c8406-ab85-4bda-acdd-bd9f145ba19c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:09.948040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:10.010200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:10.010838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:10.011119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:10.011390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:10.012983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:10.124988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:10.126535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1fff8-e6bc-41c8-abf5-38fbab5085c7 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:10.145264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 560/1123] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:12.668079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:12.668662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:12.668662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:12.669932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 564/1124] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:58:12.675365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2950e-a352-49ce-9f0b-4ee1c5f46745 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:12.675774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2950e-a352-49ce-9f0b-4ee1c5f46745 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:12.675924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2950e-a352-49ce-9f0b-4ee1c5f46745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:12.724421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:12.742038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c3b2950e-a352-49ce-9f0b-4ee1c5f46745 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:12.743271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c3b2950e-a352-49ce-9f0b-4ee1c5f46745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:12.837991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 561/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:13.126086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:13.127525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:13.230969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635c69c-a66b-4698-803c-299fb3371231 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:13.272068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:13.306002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:13.306655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:13.306936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:13.307204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:13.308816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:13.418471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:13.419916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7e51a-b4e9-40bf-a320-35617b449b12 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:13.439778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 565/1126] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:15.737466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:15.737919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:15.738014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:15.739377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 562/1127] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:58:15.744629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda521f8-5faf-4f31-989c-9047a81a2435 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:15.745045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda521f8-5faf-4f31-989c-9047a81a2435 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:15.745164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda521f8-5faf-4f31-989c-9047a81a2435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:15.811801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dda521f8-5faf-4f31-989c-9047a81a2435 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:15.813087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dda521f8-5faf-4f31-989c-9047a81a2435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:15.835162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:15.911727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 566/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:16.201501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:16.203059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:16.306089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659d9ca1-737e-496a-8d9c-47b53d7acf4f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:16.326376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:16.380256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:16.380859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:16.381136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:16.381403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:16.383019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:16.492779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:16.494203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cdb792-79e4-46e0-988e-c3c9f9f2f73f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:16.511952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 563/1129] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:18.706209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:18.706799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:18.706799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:18.708112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 567/1130] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:18.713346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31b318-15f6-472c-b400-3581e3c929b0 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:18.713725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31b318-15f6-472c-b400-3581e3c929b0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:18.713891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31b318-15f6-472c-b400-3581e3c929b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:18.778844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:18.782108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9f31b318-15f6-472c-b400-3581e3c929b0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:18.785291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9f31b318-15f6-472c-b400-3581e3c929b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:18.881984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 564/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:19.169335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:19.170920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:19.273537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729922dc-7eee-4fd0-bbbb-4454996a7920 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:19.321971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:19.346466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:19.347099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:19.347375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:19.347644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:19.349222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:19.456601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:19.458033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbc248d-d723-4a2f-a9ad-7d0cda935218 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:19.477915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 568/1132] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:21.700969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db15761-f449-4207-b038-281d9b290b5b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:21.701509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db15761-f449-4207-b038-281d9b290b5b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:21.701509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db15761-f449-4207-b038-281d9b290b5b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:21.702803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 565/1133] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:21.708613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a77d95-5831-413d-b24b-371967089b39 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:21.709055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a77d95-5831-413d-b24b-371967089b39 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:21.709159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a77d95-5831-413d-b24b-371967089b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:21.774966 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e4a77d95-5831-413d-b24b-371967089b39 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:21.776164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e4a77d95-5831-413d-b24b-371967089b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:21.805673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:21.872930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 569/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:22.167044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db15761-f449-4207-b038-281d9b290b5b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:22.168602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1db15761-f449-4207-b038-281d9b290b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:22.271152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db15761-f449-4207-b038-281d9b290b5b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:22.299560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:22.343610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:22.344811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:22.345088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:22.345371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:22.346960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:22.456684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:22.458145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a69fd-13d5-45db-947f-162b7b6fa73f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:22.476890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 566/1135] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:24.701703 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:24.702311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:24.702311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:24.704573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 570/1136] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:24.709951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ac05-1aec-443e-8a45-4d9bdbe8b40d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:24.710383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ac05-1aec-443e-8a45-4d9bdbe8b40d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:24.710493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a99ac05-1aec-443e-8a45-4d9bdbe8b40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:24.777444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0a99ac05-1aec-443e-8a45-4d9bdbe8b40d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:24.778810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0a99ac05-1aec-443e-8a45-4d9bdbe8b40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:24.788271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:24.876883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 567/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:25.171713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:25.173349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:25.276520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9dfc6-9f47-4ae3-8cb4-e0a707c74f4b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:25.332087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:25.350481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:25.351124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:25.351403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:25.351673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:25.353271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:25.462169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:25.463622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc739-da19-4f0b-a745-90491576c225 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:25.482599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 571/1138] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:27.693140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:27.693700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:27.693700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:27.695075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 568/1139] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:27.700446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d2ca-947a-436a-af33-be6565b5bf67 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:27.700852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d2ca-947a-436a-af33-be6565b5bf67 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:27.701044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa82d2ca-947a-436a-af33-be6565b5bf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:27.767204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-aa82d2ca-947a-436a-af33-be6565b5bf67 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:27.768419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-aa82d2ca-947a-436a-af33-be6565b5bf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:27.806981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:27.865684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 572/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:28.155373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:28.157447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:28.261125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55ef56-daf8-42c9-bb3c-ec9d26400edc None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:28.299396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:28.335454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:28.336053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:28.336330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:28.336600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:28.338180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:28.445520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:28.446963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b663fa-5b48-4dfd-aea7-b4106fc20e5e service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:28.465236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 569/1141] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:30.746921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:30.747498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:30.747498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:30.748814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 573/1142] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:30.754087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524d51-26b4-49bc-ac70-a9c80814d348 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:30.754530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524d51-26b4-49bc-ac70-a9c80814d348 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:30.754659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6524d51-26b4-49bc-ac70-a9c80814d348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:30.821740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f6524d51-26b4-49bc-ac70-a9c80814d348 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:30.822960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f6524d51-26b4-49bc-ac70-a9c80814d348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:30.841093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:30.920696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 570/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:31.209185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:31.210735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:31.313988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16491c3-8b01-495c-b27a-f5b807ff0efd None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:31.380079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:31.386412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:31.387045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:31.387321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:31.387588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:31.389157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:31.495823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:31.497162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49df30e9-9cb7-446e-aa23-635d3f28744c service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:31.515873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 574/1144] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:33.727499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:33.728035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:33.728035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:33.729370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 571/1145] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 14:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:33.741706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec42690-a40f-4c77-81d3-811fc2c43023 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:33.741706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec42690-a40f-4c77-81d3-811fc2c43023 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:33.741706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec42690-a40f-4c77-81d3-811fc2c43023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:33.803727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6ec42690-a40f-4c77-81d3-811fc2c43023 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:33.804942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6ec42690-a40f-4c77-81d3-811fc2c43023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:33.852352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:33.900934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 575/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 14:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:34.192905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:34.194450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:34.297953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dba578-5963-4bea-a4ea-995516ff5672 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:34.345634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:34.370927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:34.371537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:34.371816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:34.372082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:34.373675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:34.483626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:34.485290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6e811-f68c-4b19-836b-ebc77dbe54aa service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:34.503554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 572/1147] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:43.923946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=22 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:44.501284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=66 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:54.753105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff009-85eb-4884-bf46-c2af12a6194b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:54.753686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff009-85eb-4884-bf46-c2af12a6194b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:54.753686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ff009-85eb-4884-bf46-c2af12a6194b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:54.755051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 576/1148] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:54.760453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f2f3-9600-4749-a164-17c0c42569df None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:54.760879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f2f3-9600-4749-a164-17c0c42569df None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:54.760990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1f2f3-9600-4749-a164-17c0c42569df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:54.827333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a4a1f2f3-9600-4749-a164-17c0c42569df None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:54.828504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a4a1f2f3-9600-4749-a164-17c0c42569df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:54.925879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 573/1149] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:58:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:55.229241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:55.229525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:55.229803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:55.231201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 577/1150] 10.222.0.60 () {54 vars in 962 bytes} [Fri Oct 21 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:55.237028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47172c9b-4043-4ec1-a955-7902eeea4fb7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:55.237347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47172c9b-4043-4ec1-a955-7902eeea4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:55.237646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47172c9b-4043-4ec1-a955-7902eeea4fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:55.313256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-47172c9b-4043-4ec1-a955-7902eeea4fb7 None None] MFA Rules not processed for user `fccd7b631a914ab684da6d642501cb64`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:55.314467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-47172c9b-4043-4ec1-a955-7902eeea4fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:55.335096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:55.390894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 574/1151] 10.222.0.60 () {58 vars in 1059 bytes} [Fri Oct 21 14:58:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:58:55.398832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:55.400347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:55.503236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e1eb1-54b3-4f50-ad24-c8c99992934f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:55.575296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:55.575896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:55.576170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:55.576435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:55.578709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:55.631452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:55.686238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:55.688280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab2ca96-cb78-442d-a04c-da0f31047215 service glance] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:55.707364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 578/1152] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:58:57.976815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:57.977367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:57.977367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:57.978766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 575/1153] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:58:57.985038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba63efd-3c26-4a95-ac79-e20e72987985 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:57.985470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba63efd-3c26-4a95-ac79-e20e72987985 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:57.985616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba63efd-3c26-4a95-ac79-e20e72987985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:58.052611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bba63efd-3c26-4a95-ac79-e20e72987985 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:58:58.053744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bba63efd-3c26-4a95-ac79-e20e72987985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:58.150219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 579/1154] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:58:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 14:58:58.427575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:58:58.429104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:58:58.552340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d69b99f-dcf4-4bcf-b1f7-0e39652eaef6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:58:58.602329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:58:58.647056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:58:58.647662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:58:58.647939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:58:58.648210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:58:58.648843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:58:58.650275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 14:58:58.650275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 14:58:58.650275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 14:58:58.651448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e82c9b-ad6e-4ac3-8431-c634f3e2e09e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:58:58.671107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 576/1155] 10.222.0.60 () {56 vars in 1108 bytes} [Fri Oct 21 14:58:58 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 14:59:00.854852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:00.855531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:00.855531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:00.856719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 580/1156] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:59:00.862140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41970-27ac-44ba-9f8e-5f36590bf6a9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:00.862573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41970-27ac-44ba-9f8e-5f36590bf6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:00.862716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a41970-27ac-44ba-9f8e-5f36590bf6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:00.931018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b3a41970-27ac-44ba-9f8e-5f36590bf6a9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:59:00.932223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b3a41970-27ac-44ba-9f8e-5f36590bf6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:01.029145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 577/1157] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:59:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:59:01.328170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:59:01.329726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:01.339368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:01.477208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09ffa53-cc3d-4839-8b66-9e012f2b122a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:59:01.556598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:01.588382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:59:01.588932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:01.589399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:01.589894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:01.591884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:01.682234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:59:01.683854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2e2f4-7df4-462b-aef5-7ecd82815c05 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:59:01.703350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 581/1158] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:59:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:59:04.065371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:04.065892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:04.065892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:04.067237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 578/1159] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:59:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:59:04.072599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a276-039c-4981-b024-dc7da0a95b39 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:04.073012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a276-039c-4981-b024-dc7da0a95b39 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:04.073152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63a276-039c-4981-b024-dc7da0a95b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:04.141877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2e63a276-039c-4981-b024-dc7da0a95b39 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:59:04.143113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2e63a276-039c-4981-b024-dc7da0a95b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:04.188823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:04.239553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 582/1160] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:59:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:59:04.525492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:59:04.527055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:04.551952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:04.631241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3aca33-2bf0-4ae9-9b32-1b7cdd3588ed None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:59:04.703813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:59:04.704410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:04.704690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:04.704958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:04.706526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:04.778798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:04.815050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:59:04.816479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f897a11-c97e-4325-90e2-fe84e90b7425 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:59:04.834417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 579/1161] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:59:07.081554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:07.087746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:07.087746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:07.087746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 583/1162] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:59:07.088649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d574d3d-e53d-4240-a50a-ce3aeeac0477 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:07.089036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d574d3d-e53d-4240-a50a-ce3aeeac0477 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:07.089244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d574d3d-e53d-4240-a50a-ce3aeeac0477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:07.156903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3d574d3d-e53d-4240-a50a-ce3aeeac0477 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:59:07.158104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3d574d3d-e53d-4240-a50a-ce3aeeac0477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:07.253889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 580/1163] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:59:07.547340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:59:07.548889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:07.652179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e839c-8ad3-4bd5-a379-c309ff271890 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:59:07.725845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:07.727428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:59:07.728059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:07.728335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:07.728605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:07.730210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:07.838928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:59:07.840285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305575e-36d8-42ea-8dd0-d8ab70b810ab service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:59:07.857839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 584/1164] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:59:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:59:10.162210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:10.162796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:10.162796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:10.164178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 581/1165] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 14:59:10.169405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c5b5-b140-4ea6-85cc-f45fb80993d4 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:10.169822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c5b5-b140-4ea6-85cc-f45fb80993d4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:10.169965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41c5b5-b140-4ea6-85cc-f45fb80993d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:10.238355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-da41c5b5-b140-4ea6-85cc-f45fb80993d4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:59:10.239611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-da41c5b5-b140-4ea6-85cc-f45fb80993d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:10.291598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:10.335428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 585/1166] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:59:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:59:10.627547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:59:10.629088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:10.665611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:10.732525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3270ed6-ce54-4732-87e3-7c8887a421ae None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:59:10.806707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:59:10.807249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:10.807550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:10.807815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:10.809377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:10.887939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:10.917010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:59:10.918430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b181b-5c1e-4929-ae8f-2963c9dae327 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:59:10.936641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 582/1167] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:59:13.690201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:13.690753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:13.690753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:13.692106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 586/1168] 10.222.0.60 () {54 vars in 927 bytes} [Fri Oct 21 14:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 14:59:13.697600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177a01-6270-4aa8-aa5a-d080186921d5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:13.698014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177a01-6270-4aa8-aa5a-d080186921d5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:13.698151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39177a01-6270-4aa8-aa5a-d080186921d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:13.768001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-39177a01-6270-4aa8-aa5a-d080186921d5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 14:59:13.769212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-39177a01-6270-4aa8-aa5a-d080186921d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:13.864479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 583/1169] 10.222.0.60 () {58 vars in 1024 bytes} [Fri Oct 21 14:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 14:59:14.164244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 14:59:14.165834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:14.269421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6c06f-0a6d-403a-b659-34cf5cbb381b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 14:59:14.342425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 14:59:14.343193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 14:59:14.343507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 14:59:14.343738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 14:59:14.345307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 14:59:14.353768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:14.455627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 14:59:14.457067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ae014-26fb-4ccd-b1ae-472c97f0aeb7 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 14:59:14.476035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 587/1170] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 14:59:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 14:59:23.862947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=82 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 14:59:24.473698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=50 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:39.900106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:39.900635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:39.900635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:39.977694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:00:39.978924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:40.085138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 584/1171] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:00:40.092093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:40.092591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:40.092699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:40.173716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:00:40.175125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:40.272659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 588/1172] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:00:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:00:40.277280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:40.278482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:40.289988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153267a-860a-40a9-b80f-ff72014a7a20 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:40.350465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:40.383844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:40.384453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:40.384734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:40.385005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:40.385641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:00:40.385940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:40.386979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:00:40.386979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 15:00:40.386979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 15:00:40.387855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:40.399619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 585/1173] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:00:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:00:40.409040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:40.418504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a706c-c669-4e29-86dd-b3ce46f3ac2a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:40.479904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:40.512582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:40.513409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:40.513857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:40.514418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:40.515092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:40.516110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:00:40.516110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 15:00:40.516110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 15:00:40.517329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:40.628467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 589/1174] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:00:40 2022] POST /identity/v3/projects => generated 373 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:00:40.633656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:40.635226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:40.668860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc0c5a-a45d-4d01-aa77-b08dee1fe38b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:40.763462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:40.764058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:40.764332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:40.764597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:40.765222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:40.766216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:00:40.766216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 15:00:40.766216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 15:00:40.767417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:40.894549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 586/1175] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:00:40 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:00:40.899272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:40.900824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:40.964726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e3c5c2-c2c0-44b2-9924-779cda248753 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:41.049410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:41.060148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:41.060901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.061041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.061294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.061910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:41.063536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:41.076147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 590/1176] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:00:40 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:00:41.080367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:41.081869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:41.093413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302bee4-1a2c-45bd-b720-8e849e6f2728 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:41.119194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:41.189021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:41.189626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.189900 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.190168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.190827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:41.192411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e95ac8-07f8-4238-b0db-95bdf9fb9157 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:41.204937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 587/1177] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:00:41 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:00:41.209850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:41.211401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:41.245327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf48669b-b6f4-400d-9e36-834b59e9efe5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:41.339497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:41.340162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.340437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.340709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] PATH_INFO: `/v3/projects/d20bf0b4a2dd46e9a78a788cb53448ef/users/30147cf78e61482a9b483bc9176c3598/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.363388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20bf0b4a2dd46e9a78a788cb53448ef, user_id=30147cf78e61482a9b483bc9176c3598, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:41.366072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee4f24c-8f7a-4e32-bd5b-992ab181ccfe admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:41.456574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 591/1178] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:00:41 2022] PUT /identity/v3/projects/d20bf0b4a2dd46e9a78a788cb53448ef/users/30147cf78e61482a9b483bc9176c3598/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:00:41.468186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472368-9d7c-4483-a4bf-b50e616eb00a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.468474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472368-9d7c-4483-a4bf-b50e616eb00a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.472495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472368-9d7c-4483-a4bf-b50e616eb00a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.526490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-5e472368-9d7c-4483-a4bf-b50e616eb00a None None] MFA Rules not processed for user `30147cf78e61482a9b483bc9176c3598`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:00:41.527703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5e472368-9d7c-4483-a4bf-b50e616eb00a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:41.605507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 588/1179] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:00:41.618655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.618953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.619225 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.620543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 592/1180] 10.222.0.60 () {54 vars in 963 bytes} [Fri Oct 21 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:00:41.625332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fbb64-b822-4f35-976c-59c17f13b865 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.625613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fbb64-b822-4f35-976c-59c17f13b865 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.625881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377fbb64-b822-4f35-976c-59c17f13b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.658515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:41.704883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-377fbb64-b822-4f35-976c-59c17f13b865 None None] MFA Rules not processed for user `33932a27fdc0413fac63db9898ef8475`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:00:41.706086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-377fbb64-b822-4f35-976c-59c17f13b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:41.782157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 589/1181] 10.222.0.60 () {58 vars in 1060 bytes} [Fri Oct 21 15:00:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:00:41.790130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:41.791678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:41.842000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:41.894485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64861971-f9e1-43ad-a1b7-c14d1170c635 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:41.970359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:41.971001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:41.971274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:41.971540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:41.973153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:42.071883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:42.073280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:42.076849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:42.090831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 593/1182] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:00:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:00:42.140493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781151f3-f168-426b-a7b9-5fc76ffb7789 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:42.140923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781151f3-f168-426b-a7b9-5fc76ffb7789 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:42.141036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781151f3-f168-426b-a7b9-5fc76ffb7789 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:42.142286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 590/1183] 10.222.0.60 () {52 vars in 830 bytes} [Fri Oct 21 15:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:00:42.145923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:42.147171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:42.246684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedd26c-a7aa-4706-8c64-e26ee9726694 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:42.340398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:42.340950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:42.341224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:42.341493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] PATH_INFO: `/v3/users/30147cf78e61482a9b483bc9176c3598` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:42.354678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30147cf78e61482a9b483bc9176c3598)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:42.355696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:00:42.355696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 15:00:42.355696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 15:00:42.356884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:42.367016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:42.574066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30147cf78e61482a9b483bc9176c3598'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:42.574655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30147cf78e61482a9b483bc9176c3598'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:42.597974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30147cf78e61482a9b483bc9176c3598'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:42.667363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30147cf78e61482a9b483bc9176c3598'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:42.667799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30147cf78e61482a9b483bc9176c3598'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:42.688515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 594/1184] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:00:42 2022] DELETE /identity/v3/users/30147cf78e61482a9b483bc9176c3598 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:00:42.693090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:42.693387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:42.693666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:42.751051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:00:42.752251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:42.827010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:42.839127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 591/1185] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:00:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:00:42.849017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:42.850808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:42.948661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb5b53e-7c3a-419c-9dc8-eaba7b2bbd19 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:43.004124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:43.023959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:43.024586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:43.024863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:43.025135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:43.026799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:43.140159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:43.141552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373368c7-2055-43a8-8272-8b0685a28a72 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:43.160622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 595/1186] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:00:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:00:43.643624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:00:43.644869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:00:43.656852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18078c97-94e1-4b3a-bdeb-70167b08df23 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:00:43.756239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:00:43.757901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:00:43.758176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:00:43.758448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] PATH_INFO: `/v3/projects/d20bf0b4a2dd46e9a78a788cb53448ef` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:00:43.768412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20bf0b4a2dd46e9a78a788cb53448ef)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:00:43.769436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:00:43.769436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 15:00:43.769436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 15:00:43.770622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:00:43.909719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20bf0b4a2dd46e9a78a788cb53448ef', 'request_id': 'req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:43.984246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20bf0b4a2dd46e9a78a788cb53448ef', 'request_id': 'req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:43.998993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20bf0b4a2dd46e9a78a788cb53448ef', 'request_id': 'req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:44.021106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] The token cache is being invalidate because project d20bf0b4a2dd46e9a78a788cb53448ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:00:44.021396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:44.021992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d1a4df8-4e77-4198-90d2-cb3f87dc830d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:00:44.023303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 592/1187] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:00:43 2022] DELETE /identity/v3/projects/d20bf0b4a2dd46e9a78a788cb53448ef => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:00:53.158283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=62 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:00:54.019124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:02:11.843956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:11.844595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:11.844595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:11.845820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 596/1188] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 15:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 21 15:02:11.850931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d3db-ae22-4593-832e-6faa3696150f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:11.851427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d3db-ae22-4593-832e-6faa3696150f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:11.851557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc1d3db-ae22-4593-832e-6faa3696150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:11.931926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7bc1d3db-ae22-4593-832e-6faa3696150f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:11.933132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7bc1d3db-ae22-4593-832e-6faa3696150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:12.037438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 593/1189] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 15:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:12.190461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:12.195512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:12.311098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f128ff1-0661-41f4-9973-b066d1fd7ed8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:12.316528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:12.317131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:12.317411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:12.317682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:12.325749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:12.327344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569ee363-bb2a-45e5-a963-235585bc1f7c service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:12.337094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 597/1190] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:13.418036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f84e-8cd6-4cea-8ba6-c6e024667fc1 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:13.418587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f84e-8cd6-4cea-8ba6-c6e024667fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:13.418587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be87f84e-8cd6-4cea-8ba6-c6e024667fc1 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:13.419958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 594/1191] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 15:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:13.426141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad638-ed6a-426d-b0e5-bec467546bd9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:13.426621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad638-ed6a-426d-b0e5-bec467546bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:13.426759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bad638-ed6a-426d-b0e5-bec467546bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:13.446624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-18bad638-ed6a-426d-b0e5-bec467546bd9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:13.447845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-18bad638-ed6a-426d-b0e5-bec467546bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:13.455672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 598/1192] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 15:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:13.609938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f84e-8cd6-4cea-8ba6-c6e024667fc1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:13.624405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be87f84e-8cd6-4cea-8ba6-c6e024667fc1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:13.629706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:13.630294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:13.630570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:13.630868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:13.639807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:13.641386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d60a406-606b-4c45-a72b-06f948a2a5f1 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:13.651715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 595/1193] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:14.074526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dd6e6-9481-4d5a-a4f6-2032e5f45b77 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:14.074833 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dd6e6-9481-4d5a-a4f6-2032e5f45b77 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:14.075153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815dd6e6-9481-4d5a-a4f6-2032e5f45b77 None None] PATH_INFO: `/v3` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:14.076448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 599/1194] 10.222.0.16 () {54 vars in 925 bytes} [Fri Oct 21 15:02:14 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:02:14.082737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab62b-b6b4-4d2e-a86c-63504a83f90e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:14.083032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab62b-b6b4-4d2e-a86c-63504a83f90e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:14.083345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ab62b-b6b4-4d2e-a86c-63504a83f90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:14.142653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6f7ab62b-b6b4-4d2e-a86c-63504a83f90e None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:14.143864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6f7ab62b-b6b4-4d2e-a86c-63504a83f90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:14.160137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:02:14.215352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 596/1195] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Oct 21 15:02:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:17.956556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22812c1-d531-4e11-9c79-52e80892fb86 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:17.957076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22812c1-d531-4e11-9c79-52e80892fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:17.957076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22812c1-d531-4e11-9c79-52e80892fb86 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:17.958434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 600/1196] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 15:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:17.963559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7c3c-c6b3-4766-aad6-92b4c04b8923 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:17.963989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7c3c-c6b3-4766-aad6-92b4c04b8923 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:17.964144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a7c3c-c6b3-4766-aad6-92b4c04b8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:17.984258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-088a7c3c-c6b3-4766-aad6-92b4c04b8923 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:17.985462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-088a7c3c-c6b3-4766-aad6-92b4c04b8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:17.995085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 597/1197] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 15:02:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:18.002806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22812c1-d531-4e11-9c79-52e80892fb86 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:18.016285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22812c1-d531-4e11-9c79-52e80892fb86 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:18.021337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:18.021951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:18.022235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:18.022509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:18.030835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:18.032397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774e289-961d-460d-b335-ec6f0c99df51 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:18.042820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 601/1198] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:25.137626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18c7f-3556-457c-a762-ac9be5b9401d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:25.138180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18c7f-3556-457c-a762-ac9be5b9401d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:25.138180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a18c7f-3556-457c-a762-ac9be5b9401d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:25.139552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 598/1199] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 15:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:25.145753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b798c-5181-4b9f-91f8-4ffd2ef5e6c6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:25.146172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b798c-5181-4b9f-91f8-4ffd2ef5e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:25.146302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b798c-5181-4b9f-91f8-4ffd2ef5e6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:25.167709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-cf2b798c-5181-4b9f-91f8-4ffd2ef5e6c6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:25.174749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cf2b798c-5181-4b9f-91f8-4ffd2ef5e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:25.177098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 602/1200] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 15:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:25.331428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a18c7f-3556-457c-a762-ac9be5b9401d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:25.346196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a18c7f-3556-457c-a762-ac9be5b9401d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:25.351543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:25.352140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:25.353005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:25.353577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:25.361236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:25.362835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b1ea08-a04d-4330-a19d-6757066b12d9 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:25.372146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 599/1201] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:26.442036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd884d-c7a3-484c-a0b5-866f85a3c505 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:26.442621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd884d-c7a3-484c-a0b5-866f85a3c505 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:26.442621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd884d-c7a3-484c-a0b5-866f85a3c505 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:26.443997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 603/1202] 127.0.0.1 () {54 vars in 901 bytes} [Fri Oct 21 15:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:26.450959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244a55-c03d-41a7-abc6-d71f3fb11a38 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:26.451391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244a55-c03d-41a7-abc6-d71f3fb11a38 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:26.451503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86244a55-c03d-41a7-abc6-d71f3fb11a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:26.471455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-86244a55-c03d-41a7-abc6-d71f3fb11a38 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:26.472661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-86244a55-c03d-41a7-abc6-d71f3fb11a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:26.480706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 600/1203] 127.0.0.1 () {58 vars in 998 bytes} [Fri Oct 21 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:26.635154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd884d-c7a3-484c-a0b5-866f85a3c505 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:26.649739 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd884d-c7a3-484c-a0b5-866f85a3c505 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:26.655320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:26.655921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:26.656200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:26.656469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:26.665049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:26.666554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619374-9d6c-47d9-9224-d6c2edba00e0 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:26.675767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 604/1204] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:30.676053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:30.676640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:30.676640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:30.677988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 601/1205] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 15:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:30.683932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f21a6-c256-4063-8d31-a918fac216c3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:30.684354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f21a6-c256-4063-8d31-a918fac216c3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:30.684464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f21a6-c256-4063-8d31-a918fac216c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:30.704173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-457f21a6-c256-4063-8d31-a918fac216c3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:30.705374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-457f21a6-c256-4063-8d31-a918fac216c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:30.713254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 605/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 15:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:31.002225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:31.003954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:31.082419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7511b06-aa6e-4d79-8531-4c3b85bed74d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:31.086052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:31.086719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:31.087275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:31.087380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:31.096287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:31.097804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a49a3-90a8-4a3e-a240-385b72f3ac47 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:31.107293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 602/1207] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:33.253266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb1c16-2817-431c-a486-a6e8ff5c8fd9 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:33.253794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb1c16-2817-431c-a486-a6e8ff5c8fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:33.253794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb1c16-2817-431c-a486-a6e8ff5c8fd9 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:33.255102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 606/1208] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 15:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:33.260666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9befd-018b-47fd-8f00-58fcba3e188a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:33.261111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9befd-018b-47fd-8f00-58fcba3e188a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:33.261220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9befd-018b-47fd-8f00-58fcba3e188a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:33.282020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fac9befd-018b-47fd-8f00-58fcba3e188a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:33.283249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fac9befd-018b-47fd-8f00-58fcba3e188a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:33.292284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 603/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 15:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:33.581236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb1c16-2817-431c-a486-a6e8ff5c8fd9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:33.586608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=75 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:02:33.592474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb1c16-2817-431c-a486-a6e8ff5c8fd9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:33.595944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:33.596540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:33.596819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:33.597095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:33.605786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:33.607233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfbad7-7236-4b56-9568-105320a59c1a service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:33.616306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 607/1210] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:37.340732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:37.341281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:37.341281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:37.342566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 604/1211] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 15:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:37.347896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df72c4c-8190-483e-b110-a73f61bdf9a1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:37.348336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df72c4c-8190-483e-b110-a73f61bdf9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:37.348470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df72c4c-8190-483e-b110-a73f61bdf9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:37.368158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1df72c4c-8190-483e-b110-a73f61bdf9a1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:37.369351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1df72c4c-8190-483e-b110-a73f61bdf9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:37.377069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 608/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 15:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:37.680740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:37.682516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:37.747085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:02:37.760627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c5384-6d3c-4628-9e10-a80e59788e1b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:37.768120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:37.768120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:37.768120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:37.768120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:37.774679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:37.776095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b463b23-a370-4819-8a5e-6f59be414b38 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:37.785283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 605/1213] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:39.909224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:39.909752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:39.909752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:39.911148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 609/1214] 127.0.0.1 () {54 vars in 917 bytes} [Fri Oct 21 15:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:02:39.917726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d748bf-68b7-4b5e-bbe7-c10a09625df1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:39.918181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d748bf-68b7-4b5e-bbe7-c10a09625df1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:39.918282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d748bf-68b7-4b5e-bbe7-c10a09625df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:39.939229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-02d748bf-68b7-4b5e-bbe7-c10a09625df1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:02:39.940444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-02d748bf-68b7-4b5e-bbe7-c10a09625df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:39.948559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 606/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Fri Oct 21 15:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:02:40.255488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:02:40.257193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:02:40.270559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec4c7f4-3786-4f5e-ab98-9af301e421c9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:02:40.275015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:02:40.275563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:02:40.275840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:02:40.276106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:02:40.284960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:02:40.286363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600480a9-6869-4b87-9126-3bc46625d701 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:02:40.296721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 610/1216] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:02:49.934267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=11 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:02:50.292398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=14 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:03.892302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:03.892931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:03.892931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:03.922684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:03.922684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:03.934480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 607/1217] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:03.945326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:03.945905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:03.945905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:03.984306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:03.985582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.005345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 611/1218] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.010620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.026319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618661ab-e102-4da2-a125-89185fcbd939 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.032193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.037601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.055208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 608/1219] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.065491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.080046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8797a0-60aa-4a5d-98a7-6ccaf333682d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.089318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.089968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.090247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.090523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.091206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.093240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e404d-cf1c-4b8f-90a0-16e983dec272 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.251811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 612/1220] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/projects => generated 391 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:04.256779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.267998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b97be6-7d5f-4eca-9acf-e6fd8b3d3663 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.354791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.355834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.356342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.356836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.357698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.359930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0ee658-b7c8-4352-a71e-219e43c36928 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.409828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c0fc-f1e2-4231-b4d9-8ddd37e90eae None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.410172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c0fc-f1e2-4231-b4d9-8ddd37e90eae None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.410468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3177c0fc-f1e2-4231-b4d9-8ddd37e90eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.443445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3177c0fc-f1e2-4231-b4d9-8ddd37e90eae None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.444709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3177c0fc-f1e2-4231-b4d9-8ddd37e90eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.457586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 613/1221] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:04.459183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a2d4-ca8e-4756-8285-003fcf817f05 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.459461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a2d4-ca8e-4756-8285-003fcf817f05 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.459751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a9a2d4-ca8e-4756-8285-003fcf817f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.483173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 609/1222] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/users => generated 296 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:04.487717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc03e-049e-4057-8ec3-740527e05cfa None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.488305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc03e-049e-4057-8ec3-740527e05cfa None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.488827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6cc03e-049e-4057-8ec3-740527e05cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.502704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b4a9a2d4-ca8e-4756-8285-003fcf817f05 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.502704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b4a9a2d4-ca8e-4756-8285-003fcf817f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.519708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 614/1223] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.520276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ea6cc03e-049e-4057-8ec3-740527e05cfa None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.521130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.521413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.521688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.522302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ea6cc03e-049e-4057-8ec3-740527e05cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.537706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 610/1224] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.539768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284886d-25e0-49e5-8bb2-f426b40455e4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.539768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284886d-25e0-49e5-8bb2-f426b40455e4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.539992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284886d-25e0-49e5-8bb2-f426b40455e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.549750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.553578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.562809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1284886d-25e0-49e5-8bb2-f426b40455e4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.563483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 615/1225] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.564266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.564386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1284886d-25e0-49e5-8bb2-f426b40455e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.573399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 611/1226] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.574616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a620-8a92-4898-bde5-07d8f303754e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.574930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a620-8a92-4898-bde5-07d8f303754e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.575208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc7a620-8a92-4898-bde5-07d8f303754e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.581337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b3d14-5a9c-4ed9-9157-8def6859bfa0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.586349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.586993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.587271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.587542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.588844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.590578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80334359-9faf-4585-86ed-2612c1d27fb0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.595380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6bc7a620-8a92-4898-bde5-07d8f303754e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.596587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6bc7a620-8a92-4898-bde5-07d8f303754e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.600698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 616/1227] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:04.601888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04901411-ab31-4dd1-a926-2620063d8b00 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.602168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04901411-ab31-4dd1-a926-2620063d8b00 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.602443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04901411-ab31-4dd1-a926-2620063d8b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.605416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 612/1228] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.606503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73b76-c983-4fb7-81d6-2aa6d45eed5d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.606817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73b76-c983-4fb7-81d6-2aa6d45eed5d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.607095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb73b76-c983-4fb7-81d6-2aa6d45eed5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.625026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-04901411-ab31-4dd1-a926-2620063d8b00 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.626245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-04901411-ab31-4dd1-a926-2620063d8b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.626604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-deb73b76-c983-4fb7-81d6-2aa6d45eed5d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.627841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-deb73b76-c983-4fb7-81d6-2aa6d45eed5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.634145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 617/1229] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.636049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69df2-7365-496c-bc47-a0f084fb6977 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.636331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69df2-7365-496c-bc47-a0f084fb6977 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.643810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 613/1230] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.644009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b69df2-7365-496c-bc47-a0f084fb6977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.644105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.644105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.644105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.658890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.660098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.661493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d5b69df2-7365-496c-bc47-a0f084fb6977 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.662757 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d5b69df2-7365-496c-bc47-a0f084fb6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.669023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 614/1231] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.670187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.671553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 618/1232] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.672838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf194318-8764-4e49-a40d-7413ebfc86eb None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.673125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf194318-8764-4e49-a40d-7413ebfc86eb None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.673398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf194318-8764-4e49-a40d-7413ebfc86eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.683046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d451df4-dfc9-4617-aef7-ce4d7fb4f884 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.687877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.688491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.688771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.689042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.689687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.689993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.691690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.694869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bf194318-8764-4e49-a40d-7413ebfc86eb None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.696068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bf194318-8764-4e49-a40d-7413ebfc86eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.704375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 615/1233] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.708418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 619/1234] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.708583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.708692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.708692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.708692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.719868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e6c29f-f147-41e3-b3a4-56b27768ca8d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.724491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.725093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.725376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.725654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.726297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.726717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.727905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.728014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1fcee6-8882-4e02-8392-6a79680c6b50 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.736135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 620/1235] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.737006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.741425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 616/1236] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:04.741425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4fed-6fb4-45b3-a049-9f812e774c63 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.741425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4fed-6fb4-45b3-a049-9f812e774c63 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.741425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d4fed-6fb4-45b3-a049-9f812e774c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.747509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b199044-465b-431b-b2c0-f4c4b720d526 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.752124 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.752722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.752999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.753269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.753917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.754203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.755237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:04:04.755237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 15:04:04.755237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 15:04:04.756113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.760996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-211d4fed-6fb4-45b3-a049-9f812e774c63 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.762230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-211d4fed-6fb4-45b3-a049-9f812e774c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.768563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 621/1237] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.769299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.773746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 617/1238] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.775412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.776148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.776432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.779231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31d5e4a-f08d-4879-b295-0c2b43b13100 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.783769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.784357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.784641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.784912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.785549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.785849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.787460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.795829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:04.797023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:04.799971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 622/1239] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.801027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.808500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 618/1240] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:04.809400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.811450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e803c2d-6d90-4c1a-9521-464129e73ba3 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.816154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.816740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.817022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.817304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.817941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.818251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.819289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5e48-79cf-4e30-a3eb-14c52d06cd05 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.819865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.824583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.825169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.825447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.825720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.826351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.827386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:04:04.827386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 15:04:04.827386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 15:04:04.828556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.839293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 623/1241] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.839293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.842918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452942e4-f7bc-404c-b36d-b555d7853aba admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.847513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.848095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.848377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.848648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.849282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:04.849585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.851199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.863106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 624/1242] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:04.863903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.874077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703732b5-c8cb-439f-a37d-6daa279c21a1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:04.878876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:04.879699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:04.879979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:04.880254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] PATH_INFO: `/v3/projects/bbced7abbedc497f86e2ea999420b2a0/users/8632f75445ef4fd3a2f1ebbc2ed87e5b/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:04.894921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbced7abbedc497f86e2ea999420b2a0, user_id=8632f75445ef4fd3a2f1ebbc2ed87e5b, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:04.897553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:04.910261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 619/1243] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/projects => generated 391 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:04.915553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.954721 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:04.960027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 625/1244] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:04 2022] PUT /identity/v3/projects/bbced7abbedc497f86e2ea999420b2a0/users/8632f75445ef4fd3a2f1ebbc2ed87e5b/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:04.960027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:04.975196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b4f15-8a31-41c3-a939-6b81115e396f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.006454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:05.020505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b446fc-e184-4ad5-ae64-a7527b211abe admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.025115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.026810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.026810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.026810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.027118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.029018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.036641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.037256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.037535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.037808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.038451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:05.039041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.040623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.053608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 620/1245] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:05.055390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.064664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ee8502-ea02-47b2-8220-069c1b6ac109 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.069215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.069793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.070069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.070340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.070989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.073588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.111531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 626/1246] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:04 2022] POST /identity/v3/projects => generated 389 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.112378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.174445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d87a3a-0b3d-4bf5-94d0-d4a781ee68e4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.179062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.179247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 621/1247] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.179667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.179944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.180183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.180312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.180839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:05.181182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.183974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.197023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 627/1248] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:05.197023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.212588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49271eee-eba1-49a2-adf3-30f0cbd55a05 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.245564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e8794-7ae8-4851-a552-2116375207e2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.250256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.250899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.251178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.251481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.252303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.254243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.273628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.274200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.274490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.274777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.276146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.278067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99b029-8237-408e-8701-5609fa6ae3c4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.382143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 622/1249] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.382143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.424534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 628/1250] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/projects => generated 377 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.424534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.424534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.424534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.429407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:05.442276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:05.443533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:05.446903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e958bf-1cb4-42a6-a4bf-f7f6b906f8fa admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.492708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:05.503613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.505109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.505109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.505109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.505392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.506120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 629/1251] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:05.506963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.510725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.516467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdbd143-9593-4920-9c93-ac8d8900cbf7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.521761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.522372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.522672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.522948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.523602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.526089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.634918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 623/1252] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/users => generated 296 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.635248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 630/1253] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/projects => generated 383 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.635984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.636346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.647223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa36f-0a96-41aa-b6ef-834a566c7d34 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.698188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eab7cdd-8cc3-4d58-a52d-ef953b88fdab admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.702850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.703458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.703737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.704009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.704637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.705651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:04:05.705651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 15:04:05.705651 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 15:04:05.706081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.706842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.706942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.707130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.707405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.708032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.709946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.800465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 631/1254] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.801336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.810892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa86e42-e8ed-400f-8bae-2e2a8b812f2d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.815473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.816041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.816319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.816592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.817208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.819123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.826210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 624/1255] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.826210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.833788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9272714-b288-480a-9001-22ccdedd85e3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.838482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.839106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.840027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.840304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.840958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.842905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.896156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 632/1256] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.896156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.919013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 625/1257] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:05.919013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:05.919509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:05.958696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09b87-fc52-4a8a-95eb-582c5e7ec2b8 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.965098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:05.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:05.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:05.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:05.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:05.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:05.982515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7605f4-329f-4183-b028-f7dcf744c53a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:05.986203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:06.032171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.032784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.033066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.033342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.043161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.044635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.048146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 633/1258] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:05 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:06.051507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.054889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 626/1259] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:06.055601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.059484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5989d2-3bc5-4a71-9c7b-d2bb0a43143a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.065184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.065869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.066001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff178b2-822e-42fc-9ae3-4b7f900ab065 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.066273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.066610 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.067350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.069034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.070957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.071479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.071794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.072274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.072775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.075202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.084060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 634/1260] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.084060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.092565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1a7e3a-cb78-4f49-8116-a2b1e548c3ee admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.097329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.097908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.098185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.098453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.099129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.100702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.111193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 635/1261] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.111969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.121370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e7e42-aee6-463a-b548-27891c935d0e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.125829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.126382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.126682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.126951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.127608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.129126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.130384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:06.142818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 636/1262] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.142818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.150022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c86600-ff27-4480-89e9-a1646b988ee6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.155680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.156270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.156533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.156802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.157403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.159461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.172801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 627/1263] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:06 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:06.173733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.184127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58df44-3b18-46da-bd3a-d488a5e3eb8a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.189214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.189824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.190105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.190374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.191362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.192976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.204333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 628/1264] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.205234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.214950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd6c0f-23e8-4236-a175-0dab2786b28f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.219620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.220210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.220521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.220801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.221429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.223043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.233475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 629/1265] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.234100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.246197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0225ab30-7489-4a96-bf24-8f5a0c1ff1fe admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.250936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.251530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.251807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.252092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.252715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.254322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.265185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 630/1266] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.266046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.275475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06259b83-d81e-4ce0-8003-9dc79a5c35de admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.280128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.280701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.281077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.281245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.281862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.283476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.288048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 637/1267] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:06 2022] POST /identity/v3/users => generated 287 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:06.288048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.294770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 631/1268] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.295469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.298101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd204cf-04ff-4ca0-9078-6b0fc1431724 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.302910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.303456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.303739 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.304014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.304649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.304791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca14ebd4-2bcd-49b8-a824-e3d48f0df472 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.306243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.310956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.311538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.311813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.312082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.312712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.314315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.317593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 638/1269] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.319823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.324450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 632/1270] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.328794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.328918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356c0c2-80b6-450d-9443-52800abfe174 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.333600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.334433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.334743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.335018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.335650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.337326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.338266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77858e9-04fb-451a-bed2-663fa1ee0921 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.342964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.348615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.348615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.348615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.348615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.348615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.349032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 639/1271] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.349032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.358435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1c6d72-8a34-43f7-be4c-3ce09384dcfa admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.359246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 633/1272] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.363209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.363626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.364309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.364590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.364866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] PATH_INFO: `/v3/projects/b6fccf637fcd497e966bb52a7b476d7c/users/f4eb6040477f41a880e68c565edec9ed/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.377282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28120160-f2ed-4894-8112-45176f27a933 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.379403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.380065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.380347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.380625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.381322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.381807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6fccf637fcd497e966bb52a7b476d7c, user_id=f4eb6040477f41a880e68c565edec9ed, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.382990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.384444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.395053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 634/1273] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.396467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.406708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d74def-127c-4e29-b13c-3cc5724259f7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.417729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.417729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.417729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.417729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] PATH_INFO: `/v3/projects/c83d10829a81437fb4d56fe488581212/users/4593c3fd1739432fa57f4a94c65cd299/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.431224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c83d10829a81437fb4d56fe488581212, user_id=4593c3fd1739432fa57f4a94c65cd299, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.434727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.453717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 640/1274] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/b6fccf637fcd497e966bb52a7b476d7c/users/f4eb6040477f41a880e68c565edec9ed/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:06.457935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.501450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 635/1275] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/c83d10829a81437fb4d56fe488581212/users/4593c3fd1739432fa57f4a94c65cd299/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:06.505870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.520278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed93a78e-ce65-49c5-89ac-7b0aa19c6e38 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.557885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:06.570759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ffbb6f-5b79-448e-bb89-a7d276e420f8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.576740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.577517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.577807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.578078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.578757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.580404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.582735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.583712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.583712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.584093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] PATH_INFO: `/v3/projects/b0e8a6692b744a4aa6353f0247c4fcce/users/88da08adefdf40d583e73b7ff395dd8f/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.591609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 636/1276] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.592917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.599834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0e8a6692b744a4aa6353f0247c4fcce, user_id=88da08adefdf40d583e73b7ff395dd8f, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.603239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.606350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d4c70-315c-4126-b6e1-63390634409f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.611677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.612366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.612645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.612919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] PATH_INFO: `/v3/projects/d5bfb34191974243b44928b969219ede/users/33741caefb3b4d2f954f1914647b6c79/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.631121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bfb34191974243b44928b969219ede, user_id=33741caefb3b4d2f954f1914647b6c79, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.633966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.653088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 641/1277] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/b0e8a6692b744a4aa6353f0247c4fcce/users/88da08adefdf40d583e73b7ff395dd8f/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:06.654024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.679812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:06.716837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a24d7b-a882-424c-a447-0094dbb70554 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.722116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.722887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.723231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.723508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] PATH_INFO: `/v3/projects/6406e245a2fd4700aebba95795a18632/users/c82ec83bb124455693573e6c68ff12fb/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.727674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 637/1278] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/d5bfb34191974243b44928b969219ede/users/33741caefb3b4d2f954f1914647b6c79/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:06.728595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.742772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6406e245a2fd4700aebba95795a18632, user_id=c82ec83bb124455693573e6c68ff12fb, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.746217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c9027-6ad0-4d29-8866-c4e9ca671730 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.796629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f539bd-64a0-4b8e-a27d-d046ef00cac7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:06.801491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:06.802189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.802539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.802905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.803611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:06.805276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80be775b-9e70-4c53-ab7c-7af690d7a927 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:06.816580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 638/1279] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:06.817726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.818010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.818571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.842692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 642/1280] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/6406e245a2fd4700aebba95795a18632/users/c82ec83bb124455693573e6c68ff12fb/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:06.848661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6fbf-483f-473e-9781-63cd4dbc21c3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.849069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6fbf-483f-473e-9781-63cd4dbc21c3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.849354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0a6fbf-483f-473e-9781-63cd4dbc21c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.849685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] MFA Rules not processed for user `f4eb6040477f41a880e68c565edec9ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:06.851323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:06.871325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6d0a6fbf-483f-473e-9781-63cd4dbc21c3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:06.872592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6d0a6fbf-483f-473e-9781-63cd4dbc21c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:06.925683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 639/1281] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:06.929681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:06.941657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 643/1282] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:06.941657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b45f7d-dad7-4724-9200-a8e84d1a2dee None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:06.947483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b45f7d-dad7-4724-9200-a8e84d1a2dee None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:06.947483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b45f7d-dad7-4724-9200-a8e84d1a2dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:06.982424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-57b45f7d-dad7-4724-9200-a8e84d1a2dee None None] MFA Rules not processed for user `88da08adefdf40d583e73b7ff395dd8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:06.983685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57b45f7d-dad7-4724-9200-a8e84d1a2dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:06.984919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:07.001046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d0ecb-fa3e-40ff-984c-8c93589fde40 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.007238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.008562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.008562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.008739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] PATH_INFO: `/v3/projects/77156a46a45648cd8cb0ea22ab0f04a8/users/1eb3641974744d83816b6f3ad7bb6f82/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.024716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77156a46a45648cd8cb0ea22ab0f04a8, user_id=1eb3641974744d83816b6f3ad7bb6f82, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.030535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.058858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 644/1283] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.060140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53ece-0571-431a-b79a-aa7a7c1249e5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.060476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53ece-0571-431a-b79a-aa7a7c1249e5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.060701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c53ece-0571-431a-b79a-aa7a7c1249e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.088653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-78c53ece-0571-431a-b79a-aa7a7c1249e5 None None] MFA Rules not processed for user `33741caefb3b4d2f954f1914647b6c79`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:07.090430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-78c53ece-0571-431a-b79a-aa7a7c1249e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:07.097941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 640/1284] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:06 2022] PUT /identity/v3/projects/77156a46a45648cd8cb0ea22ab0f04a8/users/1eb3641974744d83816b6f3ad7bb6f82/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:07.098884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.116314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:07.156745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 645/1285] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.161423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.161423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.161423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.164287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe54fa5-fec1-4b81-9084-c67e919a19a0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.170878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.171543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.171894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.172245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] PATH_INFO: `/v3/projects/dc7f82673a5c4e44af476123af54e37f/users/e4c2c66a664b4b5d90ba77dd69fda0ac/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.180552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:07.183057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:07.191083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc7f82673a5c4e44af476123af54e37f, user_id=e4c2c66a664b4b5d90ba77dd69fda0ac, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.193527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 646/1286] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.194425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.194538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.211374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fda395-cd94-4abe-8252-8dec8875a2f3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.216731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.217404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.217688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.217961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.218652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.220590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.286839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 641/1287] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:07 2022] PUT /identity/v3/projects/dc7f82673a5c4e44af476123af54e37f/users/e4c2c66a664b4b5d90ba77dd69fda0ac/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:07.290603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.319142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 647/1288] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:07.320056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.356214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6dcf4-d0bb-4599-bcbc-4abc7ccb425f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.391654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4379e-c6b0-45d9-bf29-be26e06d7d84 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.397016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.397715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.397993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.398274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.398919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.401058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.427517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.428136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.428398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.428675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.438123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.439592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb198b8-044e-4ec2-8e8c-a736947f9553 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.450528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 642/1289] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:07.463807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0cc4-7e85-4897-9a08-c9a7d344f09e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.463807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0cc4-7e85-4897-9a08-c9a7d344f09e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.463807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a0cc4-7e85-4897-9a08-c9a7d344f09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.479559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-609a0cc4-7e85-4897-9a08-c9a7d344f09e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:07.480945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-609a0cc4-7e85-4897-9a08-c9a7d344f09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:07.492186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 643/1290] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.492186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79514534-d828-4266-80f8-7483ce725118 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.492186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79514534-d828-4266-80f8-7483ce725118 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.492186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79514534-d828-4266-80f8-7483ce725118 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.493328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 644/1291] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 15:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:07.493803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79514534-d828-4266-80f8-7483ce725118 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.506490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79514534-d828-4266-80f8-7483ce725118 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.510791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.511910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.512191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.512615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.518957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:07.522784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.524213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e2fe41-faff-4c93-84d5-9b80465266d2 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.535040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 645/1292] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:07.536166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78431dca-958d-4e25-8f53-7b24ccb77226 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.536512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78431dca-958d-4e25-8f53-7b24ccb77226 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.536787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78431dca-958d-4e25-8f53-7b24ccb77226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.538976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 648/1293] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/projects => generated 371 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:07.540069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.569417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-78431dca-958d-4e25-8f53-7b24ccb77226 None None] MFA Rules not processed for user `e4c2c66a664b4b5d90ba77dd69fda0ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:07.571065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-78431dca-958d-4e25-8f53-7b24ccb77226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:07.639242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58622d53-654a-4d1d-afb0-b55c3b753436 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.643946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.644583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.644999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.645354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.646004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.647996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.652428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 646/1294] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.654056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.654549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.655044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.706884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:07.708657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:07.755147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 649/1295] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/users => generated 303 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:07.757416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.772073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:07.799984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 647/1296] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:07.801035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.813595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a90e6-89e9-44af-8454-4e4cc2f1089f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.817939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a0e2d-899b-4fcb-815d-8bad0f84e840 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.818456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.819096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.819381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.819648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.820292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.822218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21d99d-d035-42c3-b2d9-0b6b0b1501b5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.822893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.823729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.824672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.825391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.836778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.841640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b13045-1e3b-4b8b-ab61-b28e34631731 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.857565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 650/1297] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:07.859982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6652-4aca-4065-94e3-2fcea0ebf6e3 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.860128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6652-4aca-4065-94e3-2fcea0ebf6e3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.860361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6652-4aca-4065-94e3-2fcea0ebf6e3 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.867108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 651/1298] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 15:04:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:07.868119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6652-4aca-4065-94e3-2fcea0ebf6e3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.881895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6652-4aca-4065-94e3-2fcea0ebf6e3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.887854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.888656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.888938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.889205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.889901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:07.891572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:07.903245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 652/1299] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:07.904218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:07.928116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2663034e-7502-4af5-980e-ee35331be398 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:07.937397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:07.938170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.938501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.938872 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:07.973687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 648/1300] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:07.980324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:07.980662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:07.980952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.027464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:08.028881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:08.046179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.048605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 649/1301] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 15:04:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:08.051497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.057424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.073199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 653/1302] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:08.076664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2b0609-9223-4756-b8e2-38c1841d473e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.076793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.087262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.087953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.088252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.088531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] PATH_INFO: `/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995/users/31f6e9b3304e4824ae9d2c10dda02074/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.090089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9ecb75-e65a-4a9f-825d-80c444adb731 service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.103271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.104107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.104472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.105368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.106693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.113324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.113673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b2ef67bd1f43e3be75ec12dd83b995, user_id=31f6e9b3304e4824ae9d2c10dda02074, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.115990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.128412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 654/1303] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:08.130146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.158930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b10adc-6ca3-492c-82c0-f1863c27c275 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.173761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.183807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.183807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.183807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.189366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.191858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.207887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 655/1304] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:08.208895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.222204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8fdb0-0796-4ce2-881c-d84f1b5f7a7a service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.229486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.235351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.235351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.235351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] PATH_INFO: `/v3/projects/8162be9f54e645049234d1d309c8ffd8/users/1d9ce75471c344e3bc38dec2ea2d9f7a/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.235686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 650/1305] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:08 2022] PUT /identity/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995/users/31f6e9b3304e4824ae9d2c10dda02074/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:08.236257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.255765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8162be9f54e645049234d1d309c8ffd8, user_id=1d9ce75471c344e3bc38dec2ea2d9f7a, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.263237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.266817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:08.333225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e6f748-dfda-4362-baa5-df62d65bb370 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.340341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.341007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.341297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.341573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.343985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.346198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.365100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 651/1306] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:08.366134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.381434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 656/1307] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:08 2022] PUT /identity/v3/projects/8162be9f54e645049234d1d309c8ffd8/users/1d9ce75471c344e3bc38dec2ea2d9f7a/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:08.385840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.406935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650f17-12bc-4f71-b0eb-96a1f6bd6c75 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.615393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a36c66-6f40-4ff3-9e0f-61fbcda93d03 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.622736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.623343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.623493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.623643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.624023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.624302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.624427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.624576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] PATH_INFO: `/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995/users/31f6e9b3304e4824ae9d2c10dda02074/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.625028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.631803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd24ff-ba2b-4dc4-ad44-2b185b2e7773 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.631986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29b2ef67bd1f43e3be75ec12dd83b995, user_id=31f6e9b3304e4824ae9d2c10dda02074, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.634661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.655622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 657/1308] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:08.657615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a0f5-219c-4e11-934a-8d0e24e0fb00 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.657982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a0f5-219c-4e11-934a-8d0e24e0fb00 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.658404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8a0f5-219c-4e11-934a-8d0e24e0fb00 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.660079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 652/1309] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:08 2022] PUT /identity/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995/users/31f6e9b3304e4824ae9d2c10dda02074/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:08.660977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:08.673149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 658/1310] 10.222.0.60 () {54 vars in 912 bytes} [Fri Oct 21 15:04:08 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:08.675022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3e3-e7ad-4b1c-8d41-f47f262b0b5c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.675589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3e3-e7ad-4b1c-8d41-f47f262b0b5c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.676916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:08.677581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b650d3e3-e7ad-4b1c-8d41-f47f262b0b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.727366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b650d3e3-e7ad-4b1c-8d41-f47f262b0b5c None None] MFA Rules not processed for user `31f6e9b3304e4824ae9d2c10dda02074`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:08.728915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b650d3e3-e7ad-4b1c-8d41-f47f262b0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:08.855885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8585aa-0da5-4e1f-a4e8-6b0b1af0b703 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:08.855885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:08.855885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.855885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.856481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] PATH_INFO: `/v3/projects/8162be9f54e645049234d1d309c8ffd8/users/1d9ce75471c344e3bc38dec2ea2d9f7a/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.856481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8162be9f54e645049234d1d309c8ffd8, user_id=1d9ce75471c344e3bc38dec2ea2d9f7a, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:08.856481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e9a18b-2f50-4f95-881b-ba97a204a98e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:08.878591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 653/1311] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:08 2022] PUT /identity/v3/projects/8162be9f54e645049234d1d309c8ffd8/users/1d9ce75471c344e3bc38dec2ea2d9f7a/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:08.880262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde2b4-206f-4997-ae2d-8255535c98ab None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.880712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde2b4-206f-4997-ae2d-8255535c98ab None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.880829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fde2b4-206f-4997-ae2d-8255535c98ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.914250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-81fde2b4-206f-4997-ae2d-8255535c98ab None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:08.918795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-81fde2b4-206f-4997-ae2d-8255535c98ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:08.931606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 659/1312] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:08.933022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd86-88ec-4295-9bc4-c00d20e5392e None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.934112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd86-88ec-4295-9bc4-c00d20e5392e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.934473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ffd86-88ec-4295-9bc4-c00d20e5392e None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.941892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 660/1313] 10.222.0.60 () {54 vars in 912 bytes} [Fri Oct 21 15:04:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:08.944282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:08.944826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:08.945130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:08.987077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] MFA Rules not processed for user `1d9ce75471c344e3bc38dec2ea2d9f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:08.988720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:09.069096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:09.095096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 654/1314] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:09.096891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:09.097285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:09.097663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:09.112106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 661/1315] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:09.114411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:09.125130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:09.126619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:09.147601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 655/1316] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:04:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:09.150106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:09.233661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f24a0-39ac-4db4-9748-8d531de87f15 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:09.241228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:09.244824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:09.247371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:09.248470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:09.289997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a00d0-a670-47ac-8788-5c6248183293 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:09.295832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:09.298280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:09.298682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:09.299028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:09.310959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:09.312515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:09.330390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 656/1317] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:09.362034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:09.365653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:09.383775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 662/1318] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:09.476516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:09.489359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29993fc-4f36-473c-a108-920dca89b1dd service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:09.499932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:09.503233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:09.503623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:09.504939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:09.525165 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:09.526776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:09.541553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 657/1319] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:09.697618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:09.710046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e16793-296c-4036-8f48-ad87e5b51df0 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:09.718875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:09.720292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:09.720292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:09.720292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:09.752122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:09.758976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:09.773603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 663/1320] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:09.806878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:10.566504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:10.578078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7d7b9-8575-4d44-b708-0f261667505d service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:10.582414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:10.583260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:10.585940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:10.585940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:10.702952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:10.706375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:10.732941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 658/1321] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:10 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:11.226975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:11.250529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176010c-ee40-494d-aac1-6e99b7a459e2 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:11.259810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:11.263045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:11.264461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:11.264897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:11.404092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:11.406114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa313f-32e1-438a-afda-4c2913d95517 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:11.433916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 664/1322] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:15.513490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:15.641833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:15.658290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057b34-3ce1-41a2-9366-6486072524c0 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:15.665107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:15.667268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:15.672254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:15.672254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] PATH_INFO: `/v3/users/e4c2c66a664b4b5d90ba77dd69fda0ac` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:15.672254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c2c66a664b4b5d90ba77dd69fda0ac)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:15.679455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: WARNING py.warnings [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:04:15.679455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: warnings.warn(msg) Oct 21 15:04:15.679455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: Oct 21 15:04:15.684404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:16.101694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c2c66a664b4b5d90ba77dd69fda0ac'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:16.143642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c2c66a664b4b5d90ba77dd69fda0ac'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:16.145001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c2c66a664b4b5d90ba77dd69fda0ac'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:16.165521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c2c66a664b4b5d90ba77dd69fda0ac'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:16.166580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c2c66a664b4b5d90ba77dd69fda0ac'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:16.270368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 659/1323] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:15 2022] DELETE /identity/v3/users/e4c2c66a664b4b5d90ba77dd69fda0ac => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:16.275041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:16.275968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:16.276398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:16.302849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:16.304458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:16.474370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 665/1324] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:16.505065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:16.506900 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:16.639478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4825b2-bb22-4e9d-a1bf-5580f2b4edf5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:16.645002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:16.646259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:16.646544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:16.646866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:16.658303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:16.659844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd06ce7-4931-45a4-8057-e080b410d66d service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:16.680768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 660/1325] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:17.618056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:17.621034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:17.667580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1cc85-2166-48ff-bda8-144c4250dbcb None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:17.668322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd905-8a80-4594-ad5c-40d54b7ac033 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:17.669519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd905-8a80-4594-ad5c-40d54b7ac033 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:17.669519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543dd905-8a80-4594-ad5c-40d54b7ac033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:17.680545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:17.682619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:17.684625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:17.685078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] PATH_INFO: `/v3/projects/dc7f82673a5c4e44af476123af54e37f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:17.688720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc7f82673a5c4e44af476123af54e37f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:17.692688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: WARNING py.warnings [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 21 15:04:17.692688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: warnings.warn(msg) Oct 21 15:04:17.692688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: Oct 21 15:04:17.697448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:17.769213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-543dd905-8a80-4594-ad5c-40d54b7ac033 None None] MFA Rules not processed for user `8632f75445ef4fd3a2f1ebbc2ed87e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:17.772652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-543dd905-8a80-4594-ad5c-40d54b7ac033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:17.853185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:17.914911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc7f82673a5c4e44af476123af54e37f', 'request_id': 'req-5187f5a4-d836-4738-b202-b337cc2af0a2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:18.013325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc7f82673a5c4e44af476123af54e37f', 'request_id': 'req-5187f5a4-d836-4738-b202-b337cc2af0a2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:18.032334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc7f82673a5c4e44af476123af54e37f', 'request_id': 'req-5187f5a4-d836-4738-b202-b337cc2af0a2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:18.082419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 661/1326] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:18.091941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] The token cache is being invalidate because project dc7f82673a5c4e44af476123af54e37f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:18.092220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:18.092948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5187f5a4-d836-4738-b202-b337cc2af0a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:18.098596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 666/1327] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:17 2022] DELETE /identity/v3/projects/dc7f82673a5c4e44af476123af54e37f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:18.111942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb57d5b-1801-48a1-bb25-4d71c29eb703 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.112131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.112131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.112351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb57d5b-1801-48a1-bb25-4d71c29eb703 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.112464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.112561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb57d5b-1801-48a1-bb25-4d71c29eb703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.114316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:18.145757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] MFA Rules not processed for user `8632f75445ef4fd3a2f1ebbc2ed87e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:18.152184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:18.170248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7fb57d5b-1801-48a1-bb25-4d71c29eb703 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:18.172543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7fb57d5b-1801-48a1-bb25-4d71c29eb703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:18.274577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 662/1328] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:18.286557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:18.288379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:18.329159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 667/1329] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:18.336728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.337115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.337466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.378441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:18.379805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:18.414260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 668/1330] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:18.434769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:18.454562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792eb229-191e-41ee-9179-8cca3553b5bd None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:18.478236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:18.479015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.479395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.481172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.487541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:18.487936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:18.489928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:18.498324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4888788-6c11-40a3-8d5c-f3da6b5359f4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:18.515390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:18.516027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.519121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.519522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.524381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 669/1331] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:18.529608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:18.529769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:18.552228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa2f34-76e3-46c0-ab7b-a847656eb4c4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:18.563742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:18.566255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:18.566926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.567207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.571840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.571840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:18.572173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:18.572257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:18.593046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 663/1332] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:18.718394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 670/1333] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:18 2022] POST /identity/v3/projects => generated 401 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:18.722000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:18.739792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e8960-1c79-4ec3-85de-f5d42a2aa9db service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:18.991880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:18.992931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:18.993677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:18.994125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:18.994958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:18.997747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:19.110792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 664/1334] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:18 2022] POST /identity/v3/users => generated 301 bytes in 389 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:19.115145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:19.131648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eb2fce-d656-42cf-9519-7ee66a5ffd60 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:19.144089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:19.144863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.145328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.145776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.146699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:19.156055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:19.186138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 671/1335] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:19.187521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:19.214789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0db7ab-c794-4f12-a687-74391d7229c8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:19.224631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:19.234496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.234496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.234496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.234496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:19.235838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b30aa0-e0d3-4bb3-a0f8-3e6a06658d59 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:19.269449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 665/1336] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:19.270362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:19.289200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150d1e5-e124-486e-9a02-aacf32dc1597 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:19.317878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:19.318516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.318948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.319226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] PATH_INFO: `/v3/projects/d4bcbf9e774d4c6fa12949219c837cfe/users/8952fcb678b04b0ba580f737f5c6bbd4/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.384788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4bcbf9e774d4c6fa12949219c837cfe, user_id=8952fcb678b04b0ba580f737f5c6bbd4, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:19.388067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:19.510110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 672/1337] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:19 2022] PUT /identity/v3/projects/d4bcbf9e774d4c6fa12949219c837cfe/users/8952fcb678b04b0ba580f737f5c6bbd4/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:19.522282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.522565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.522881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.558910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:19.561919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:19.578561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] MFA Rules not processed for user `8952fcb678b04b0ba580f737f5c6bbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:19.579854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:19.583474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:19.672947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:19.752583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521fa491-6680-4a08-a586-340a01baf2ce admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:19.769191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:19.773339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.773627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.775494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] PATH_INFO: `/v3/users/f4eb6040477f41a880e68c565edec9ed` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.780729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4eb6040477f41a880e68c565edec9ed)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:19.791291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:19.801304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 666/1338] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:19.807142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:19.845829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0e9a5-3ccb-4808-8b29-1eef25022b28 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:19.871282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:19.872437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:19.872738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:19.873024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:19.873761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:19.875769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83ee6b-4e9f-49ab-9ee6-6f3aaee55018 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.020129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 667/1339] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:19 2022] POST /identity/v3/projects => generated 399 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:20.021674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7931-360a-4c2f-9420-88dfada1dc59 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.022057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7931-360a-4c2f-9420-88dfada1dc59 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.022332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa7931-360a-4c2f-9420-88dfada1dc59 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.026300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 668/1340] 10.222.0.60 () {54 vars in 912 bytes} [Fri Oct 21 15:04:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:20.034921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa7931-360a-4c2f-9420-88dfada1dc59 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.096218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4eb6040477f41a880e68c565edec9ed'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:20.097984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4eb6040477f41a880e68c565edec9ed'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:20.126772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4eb6040477f41a880e68c565edec9ed'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:20.218472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4eb6040477f41a880e68c565edec9ed'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:20.218724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8cd3ba2f-5e6d-43b3-98e8-b2bceb59c87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4eb6040477f41a880e68c565edec9ed'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:20.233148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa7931-360a-4c2f-9420-88dfada1dc59 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:20.239970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:20.240191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 673/1341] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:19 2022] DELETE /identity/v3/users/f4eb6040477f41a880e68c565edec9ed => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:20.241218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.241639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.242091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.243835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:20.244259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.245550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.245836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.246918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ad76de-d41f-425a-8f52-870d6182b3e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.325094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:20.325706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:20.384673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 669/1342] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:20 2022] POST /identity/v3/users => generated 306 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:20.390786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.391077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.391357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.434154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 674/1343] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:20.435522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.441520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:20.442234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:20.443665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:20.600144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 670/1344] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:20.601063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.612677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:20.614024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a204e76-bc24-4a27-9ec8-53710876f01b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:20.626559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:20.635738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.635738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.635738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.635738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:20.635738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.649463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc106fb-0109-4adb-a81c-7c8ccbbec964 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:20.666797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 675/1345] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:20.676391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.676502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:20.676502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.676502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.676502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.692654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e2d8b-f50b-4e2b-8415-2902fcaf3402 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:20.697722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:20.704527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.707361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:20.710222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.710222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.710222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] PATH_INFO: `/v3/projects/e1dc7834d55848089160cf7752013f25/users/fb45028753ee4a1bb10c8dd6ada5c8b7/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.721779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 671/1346] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:20.724999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.744660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee5a2-0d11-4ad9-aab6-18a9c3a20cd3 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:20.745727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dc7834d55848089160cf7752013f25, user_id=fb45028753ee4a1bb10c8dd6ada5c8b7, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:20.748613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.760552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:20.761745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:20.762290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:20.762814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:20.784939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:20.787180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:20.799710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:20.808680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 672/1347] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:20.864641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 676/1348] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:20 2022] PUT /identity/v3/projects/e1dc7834d55848089160cf7752013f25/users/fb45028753ee4a1bb10c8dd6ada5c8b7/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:20.876551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:20.879943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d67efe-baa8-4821-bca1-1c56f5cc0151 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:21.031071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:21.031706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.032001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.032861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.033561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:21.039420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc1c721-266d-440a-a041-1a549b0396f6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:21.074661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 673/1349] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:21.076470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:21.091192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f320849-dbce-4618-85b1-1880a6c6fc3e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:21.103999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:21.105435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.105874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.106478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] PATH_INFO: `/v3/projects/e1dc7834d55848089160cf7752013f25/users/fb45028753ee4a1bb10c8dd6ada5c8b7/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.110813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1dc7834d55848089160cf7752013f25, user_id=fb45028753ee4a1bb10c8dd6ada5c8b7, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:21.114883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943db56f-f73d-462c-9136-f72a0ecdb28e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:21.147308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 677/1350] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:21 2022] PUT /identity/v3/projects/e1dc7834d55848089160cf7752013f25/users/fb45028753ee4a1bb10c8dd6ada5c8b7/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:21.150077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.150472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.150883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.229430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] MFA Rules not processed for user `fb45028753ee4a1bb10c8dd6ada5c8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:21.234718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:21.368201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.368606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.368933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.374082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 674/1351] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:21.387907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:21.391289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:21.444247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] MFA Rules not processed for user `1eb3641974744d83816b6f3ad7bb6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:21.445657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:21.514959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:21.542962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af4041-f20b-48f5-ba8e-91bb061b5da0 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:21.554600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:21.555783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.556232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.556669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.583942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:21.593023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:21.616714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 678/1352] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:21.642543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:21.644159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:21.661192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 675/1353] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:21.817973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6044d92-b0b8-4fe3-9e37-dfdda4d50b6f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:21.825706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:21.827285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.827755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.828233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.848297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:21.850197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:21.880609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 679/1354] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:21.906250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:21.920563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9474411-977b-460c-84c6-70120371df32 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:21.937847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:21.941049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:21.941049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:21.941049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:21.947821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:21.965461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:22.131998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:22.133962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cea752-d2d3-4a22-a918-34030c8e4ed5 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:22.157562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 676/1355] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:21 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:22.365395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:22.368080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:22.426604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:22.427127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:22.427557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:22.538570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] MFA Rules not processed for user `4593c3fd1739432fa57f4a94c65cd299`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:22.540404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:22.615298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f531d9-52b9-484a-847c-90f564047f7f service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:22.628320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:22.634328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:22.638866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:22.639158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] PATH_INFO: `/v3/projects/b6fccf637fcd497e966bb52a7b476d7c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:22.642765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6fccf637fcd497e966bb52a7b476d7c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:22.653450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:22.704045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 677/1356] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:22 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:22.704045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:22.705401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:22.843956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da85d0-8a26-4074-87b2-cea6609e1535 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:22.852100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:22.853498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:22.853887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:22.854281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:22.860875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:22.876603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:22.879597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:22.881312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6fccf637fcd497e966bb52a7b476d7c', 'request_id': 'req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:22.893712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 678/1357] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:22.895517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:22.910817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add55b2a-2cc9-4617-9a6c-9d19a456aa83 service placement] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:22.937498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:23.029517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6fccf637fcd497e966bb52a7b476d7c', 'request_id': 'req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:23.042332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6fccf637fcd497e966bb52a7b476d7c', 'request_id': 'req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:23.055335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:23.071132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:23.071516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:23.071828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:23.093201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] The token cache is being invalidate because project b6fccf637fcd497e966bb52a7b476d7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:23.101220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:23.115200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-fc7846ff-5f8f-4080-9bff-1ce2c1bede09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:23.120375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 680/1358] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:22 2022] DELETE /identity/v3/projects/b6fccf637fcd497e966bb52a7b476d7c => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:23.121634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92574dc-3ad5-4af3-915e-c2498893a4c6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:23.125661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92574dc-3ad5-4af3-915e-c2498893a4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:23.125661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92574dc-3ad5-4af3-915e-c2498893a4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:23.213905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a92574dc-3ad5-4af3-915e-c2498893a4c6 None None] MFA Rules not processed for user `8632f75445ef4fd3a2f1ebbc2ed87e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:23.223423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a92574dc-3ad5-4af3-915e-c2498893a4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:23.316245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:23.318210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:23.730928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 679/1359] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:23.746799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:23.748760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:23.748995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 681/1360] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:23.749664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95613f8-01b9-4d19-aef2-59a34dbab030 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:23.749664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:23.749664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:23.749664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:23.750047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] PATH_INFO: `/v3/users/31f6e9b3304e4824ae9d2c10dda02074` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:23.750047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31f6e9b3304e4824ae9d2c10dda02074)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:23.750047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:23.750543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df088a2-33ef-4b31-abc5-45568fd66c6f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:23.750543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df088a2-33ef-4b31-abc5-45568fd66c6f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:23.750543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df088a2-33ef-4b31-abc5-45568fd66c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:23.831449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1df088a2-33ef-4b31-abc5-45568fd66c6f None None] MFA Rules not processed for user `c82ec83bb124455693573e6c68ff12fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:23.834716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1df088a2-33ef-4b31-abc5-45568fd66c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:24.030284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31f6e9b3304e4824ae9d2c10dda02074'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.062952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:24.067004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31f6e9b3304e4824ae9d2c10dda02074'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.067787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31f6e9b3304e4824ae9d2c10dda02074'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.087333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 682/1361] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:23 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 375 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:24.089003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:24.095331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:24.095331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:24.104027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31f6e9b3304e4824ae9d2c10dda02074'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.104027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31f6e9b3304e4824ae9d2c10dda02074'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.136756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] MFA Rules not processed for user `c82ec83bb124455693573e6c68ff12fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:24.139962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:24.168400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 683/1362] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:24.174864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:24.196557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:24.244173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 680/1363] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:23 2022] DELETE /identity/v3/users/31f6e9b3304e4824ae9d2c10dda02074 => generated 0 bytes in 889 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:24.244173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:24.253643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ed4c2-afeb-4312-9988-b4659e0ce9e6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:24.255297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:24.472631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cd354-c4ec-4c80-a5f7-b9f98d9d96f6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:24.479400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:24.479400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:24.479400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:24.479400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] PATH_INFO: `/v3/users/33741caefb3b4d2f954f1914647b6c79` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:24.483644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=33741caefb3b4d2f954f1914647b6c79)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:24.489950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:24.527255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:24.527896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:24.528180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:24.528456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:24.531617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:24.680488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:24.681886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104c2321-5254-4a8d-9f06-89e860026b38 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:24.712483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 681/1364] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:24.833198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33741caefb3b4d2f954f1914647b6c79'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.844409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33741caefb3b4d2f954f1914647b6c79'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:24.876296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33741caefb3b4d2f954f1914647b6c79'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:25.103195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33741caefb3b4d2f954f1914647b6c79'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:25.103195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3592f86-2624-45a6-b3a1-99d243d8ea7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33741caefb3b4d2f954f1914647b6c79'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:25.112706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 684/1365] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:24 2022] DELETE /identity/v3/users/33741caefb3b4d2f954f1914647b6c79 => generated 0 bytes in 944 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:25.117377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:25.117861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:25.117932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:25.164526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:25.168193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:25.324751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0131d3-a1cc-4506-b68e-7524a124612b None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:25.325050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0131d3-a1cc-4506-b68e-7524a124612b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:25.325375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0131d3-a1cc-4506-b68e-7524a124612b None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:25.326916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 685/1366] 10.222.0.60 () {54 vars in 912 bytes} [Fri Oct 21 15:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:25.332745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:25.333032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:25.333334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:25.371071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 682/1367] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:25.384063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:25.385752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:25.393065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:25.395686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:25.571914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb819416-b594-45e6-b9c8-33bf3aae7166 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:25.580825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:25.601409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:25.601409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:25.601409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:25.652602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 686/1368] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:04:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:25.670820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:25.670820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:25.679792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:25.709996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 683/1369] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:25.742290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:25.743726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e286dfc-520c-4d75-b0fb-cc23f4af3473 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:25.765606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:25.770670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:25.770977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:25.771260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:25.809487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:25.815537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ce56ec-b5c2-49c9-abfa-fb2da1825e3d service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:25.879498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 687/1370] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:04:25.998255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:26.000848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:26.044370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a17d1-87a8-4092-b124-d0530568131b service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:26.076229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:26.076938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:26.077208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:26.077485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] PATH_INFO: `/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:26.091622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=29b2ef67bd1f43e3be75ec12dd83b995)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:26.110606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:26.212854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:26.378106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29b2ef67bd1f43e3be75ec12dd83b995', 'request_id': 'req-746f4fad-ac1a-4adb-8a60-7a796437f65c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:26.571912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29b2ef67bd1f43e3be75ec12dd83b995', 'request_id': 'req-746f4fad-ac1a-4adb-8a60-7a796437f65c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:26.579669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29b2ef67bd1f43e3be75ec12dd83b995', 'request_id': 'req-746f4fad-ac1a-4adb-8a60-7a796437f65c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:26.607769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] The token cache is being invalidate because project 29b2ef67bd1f43e3be75ec12dd83b995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:26.608181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:26.609517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-746f4fad-ac1a-4adb-8a60-7a796437f65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:26.612678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 684/1371] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:25 2022] DELETE /identity/v3/projects/29b2ef67bd1f43e3be75ec12dd83b995 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:26.617400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:26.617919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:26.618107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:26.701880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:26.703195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:26.904799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 688/1372] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:26.917971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:26.918498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:26.919009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:26.974777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:26.983836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:27.012355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 685/1373] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:27.017020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.055481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4943d-6ca9-4939-9c0b-af1c1f2dcb9b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.095433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.096388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.096813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.097230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.098120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:27.099743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.102018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:27.147093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 689/1374] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:27.151303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.168127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71cb5f-9daa-46c3-8eec-fc8099b99711 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.181814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.183667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.184100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.184505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.185353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.188501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:27.343725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 686/1375] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:27 2022] POST /identity/v3/projects => generated 383 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:27.358476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.373790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ab1e3-b77b-4619-a359-008a55fa9546 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.576062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.580964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.581766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.582969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.583800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.595700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:27.781061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 690/1376] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:27 2022] POST /identity/v3/users => generated 292 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:27.788517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.819096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849612ef-019b-4481-ba14-7c3c6f032e81 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.826021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.827055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.827539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.828066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.829580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.831773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:27.850250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 687/1377] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:27.855736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.870976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b14aa9a-c8af-456d-b087-58baf5f7701e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.883792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.884664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.885100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.886114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.887433 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.892238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:27.910540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 691/1378] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:27 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:27.915012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:27.931403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ca0f1-b290-46d8-b2be-f6f65737d3d6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:27.944007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:27.944007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:27.944007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:27.944007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] PATH_INFO: `/v3/projects/5eb130530aa4458087264044c21e17ad/users/7aad33b912b94cbf98948612bbe3e417/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:27.961288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eb130530aa4458087264044c21e17ad, user_id=7aad33b912b94cbf98948612bbe3e417, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:27.965596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56493f0-b3ea-4109-85a8-b4aa7d195979 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:28.028063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:28.028063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:28.061678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dec39e-588e-486a-95a6-2e44ec6f07a1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:28.075895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:28.077161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.078879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.080141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] PATH_INFO: `/v3/projects/d5bfb34191974243b44928b969219ede` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.083858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bfb34191974243b44928b969219ede)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:28.122150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:28.150450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 688/1379] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:27 2022] PUT /identity/v3/projects/5eb130530aa4458087264044c21e17ad/users/7aad33b912b94cbf98948612bbe3e417/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:28.160902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292106-715b-435f-8e89-eeca811674a2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.161342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292106-715b-435f-8e89-eeca811674a2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.161778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14292106-715b-435f-8e89-eeca811674a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.258022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-14292106-715b-435f-8e89-eeca811674a2 None None] MFA Rules not processed for user `7aad33b912b94cbf98948612bbe3e417`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:28.259443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-14292106-715b-435f-8e89-eeca811674a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:28.350593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bfb34191974243b44928b969219ede', 'request_id': 'req-9e35358d-bab4-4849-9211-cd961852a41d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:28.452312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bfb34191974243b44928b969219ede', 'request_id': 'req-9e35358d-bab4-4849-9211-cd961852a41d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:28.471152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bfb34191974243b44928b969219ede', 'request_id': 'req-9e35358d-bab4-4849-9211-cd961852a41d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:28.505558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] The token cache is being invalidate because project d5bfb34191974243b44928b969219ede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:28.506160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:28.507042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9e35358d-bab4-4849-9211-cd961852a41d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:28.527904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 692/1380] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:28 2022] DELETE /identity/v3/projects/d5bfb34191974243b44928b969219ede => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:28.531530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b54e0-2626-4146-96d7-cd9066768f6d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.531904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b54e0-2626-4146-96d7-cd9066768f6d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.532853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b54e0-2626-4146-96d7-cd9066768f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.590807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2d3b54e0-2626-4146-96d7-cd9066768f6d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:28.592327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2d3b54e0-2626-4146-96d7-cd9066768f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:28.656049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 689/1381] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 496 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:28.657021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292106-715b-435f-8e89-eeca811674a2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:28.661323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-14292106-715b-435f-8e89-eeca811674a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:28.773682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 693/1382] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:28.782299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.782873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.782873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.844038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14292106-715b-435f-8e89-eeca811674a2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:28.850826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:28.851535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.852198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.852676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.857921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:28.860620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:28.863661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:28.864891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:28.879866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 694/1383] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:28.884001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:28.896113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11707197-5b7b-4a5f-be6b-a95de168d926 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:28.918167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:28.920399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:28.920897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:28.921902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:28.922848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:28.924294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:28.926942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:28.968861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 695/1384] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:04:28.974051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.009272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d30774-cb57-452c-940f-c4602a95784a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.020403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.023769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.023769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.023769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.027641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 690/1385] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:28 2022] POST /identity/v3/projects => generated 381 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:29.032261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.040707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.055470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.218921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 696/1386] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:28 2022] POST /identity/v3/projects => generated 365 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:29.223046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.353495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012e9520-12cc-4b71-8cf6-0ded96355967 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.553821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37a37b-0beb-46cb-9f48-90292142a9cf admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.554486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.555956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.556499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.556779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.557502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.559826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.603408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.605047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.607113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.607513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.608244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.610710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.687027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:29.695688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 691/1387] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:29 2022] POST /identity/v3/users => generated 297 bytes in 664 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:29.703173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.747890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9135e9d-f3e2-463c-94fe-6b6647e4376f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.753760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.755089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.755450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.756132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.757144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.759403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.766274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 697/1388] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:29 2022] POST /identity/v3/users => generated 283 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:29.768058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.775181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:29.789947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 692/1389] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:29.791803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:29.805936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407dbb65-3118-4381-a743-498c67cc652c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.809233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2fdab6-d86a-4335-a949-6f911948dc88 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:29.816713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.817603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.818080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.818851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.819702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.822221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a2c66a-3441-46c9-bf72-7a986eed213f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.830056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:29.832254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.834558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.861409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] PATH_INFO: `/v3/users/8952fcb678b04b0ba580f737f5c6bbd4` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.861409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8952fcb678b04b0ba580f737f5c6bbd4)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:29.861552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 693/1390] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:29.861552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:29.861552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:29.861552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:29.861829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:29.932492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:29.934969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.044778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 694/1391] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:04:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:30.045899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.060244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5b566-3695-4adc-9077-94b72b911ccb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:30.077161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:30.078178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.078317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.082882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] PATH_INFO: `/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc/users/d50dfcf30325413381ac3a1dc39e7f99/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.117591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fee6be7054a4e55b7f374e4c1c77ccc, user_id=d50dfcf30325413381ac3a1dc39e7f99, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:30.128091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:30.190344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8952fcb678b04b0ba580f737f5c6bbd4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:30.191278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8952fcb678b04b0ba580f737f5c6bbd4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:30.210945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:30.234163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8952fcb678b04b0ba580f737f5c6bbd4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:30.270288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 695/1392] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:30 2022] PUT /identity/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc/users/d50dfcf30325413381ac3a1dc39e7f99/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:30.270947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.364061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45125-09e9-4d0b-a8ba-bf1a353de2ea admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:30.366836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8952fcb678b04b0ba580f737f5c6bbd4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:30.367632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2e3a2225-70aa-4e6e-855e-2516f344efa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8952fcb678b04b0ba580f737f5c6bbd4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:30.373254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:30.374180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.374832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.377497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.378683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:30.380684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:30.399943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 696/1393] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:30.400311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.404989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 698/1394] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:29 2022] DELETE /identity/v3/users/8952fcb678b04b0ba580f737f5c6bbd4 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:30.410462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685e8c4-cfdd-409a-898d-9d638c877c9e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.411669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.414830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685e8c4-cfdd-409a-898d-9d638c877c9e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.416406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d685e8c4-cfdd-409a-898d-9d638c877c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.445529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de3703-b24e-4624-81ab-645cbcb7e249 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:30.452616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d685e8c4-cfdd-409a-898d-9d638c877c9e None None] MFA Rules not processed for user `4593c3fd1739432fa57f4a94c65cd299`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:30.453939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d685e8c4-cfdd-409a-898d-9d638c877c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.534340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:30.535004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.535280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.535547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.536186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:30.537933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:30.553132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 697/1395] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:30.553132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.555802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.567764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 699/1396] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:30.569875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b394-5e25-4a6c-897b-e5f6a8da8167 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.572693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b394-5e25-4a6c-897b-e5f6a8da8167 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.575284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493b394-5e25-4a6c-897b-e5f6a8da8167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.591404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80537701-4778-4ff2-bf8b-76f1f0dde8ad admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:30.604676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:30.609157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.609348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8493b394-5e25-4a6c-897b-e5f6a8da8167 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:30.609989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.610656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] PATH_INFO: `/v3/projects/d5c6293e30b94963ada751aba24a570f/users/bb5ea3edc43743279902a002b8fe098c/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.611543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8493b394-5e25-4a6c-897b-e5f6a8da8167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.621463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 700/1397] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:30.625816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:30.626308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:30.626747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:30.658863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5c6293e30b94963ada751aba24a570f, user_id=bb5ea3edc43743279902a002b8fe098c, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:30.665241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] MFA Rules not processed for user `1eb3641974744d83816b6f3ad7bb6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:30.669759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.671442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:30.758833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 698/1398] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:30 2022] PUT /identity/v3/projects/d5c6293e30b94963ada751aba24a570f/users/bb5ea3edc43743279902a002b8fe098c/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:30.758833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.768138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:30.836775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 701/1399] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:30.844277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:30.898370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ba0e3b-582e-4f21-8519-6990d4304562 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:31.033682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec9f7c-6bb4-4d9c-aa9f-458a73ced4b2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:31.042074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:31.042826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.043293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.043735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] PATH_INFO: `/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc/users/d50dfcf30325413381ac3a1dc39e7f99/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.044479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:31.046041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fee6be7054a4e55b7f374e4c1c77ccc, user_id=d50dfcf30325413381ac3a1dc39e7f99, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:31.048471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.048769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.049114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.049392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:31.055172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.063292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 702/1400] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:30 2022] PUT /identity/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc/users/d50dfcf30325413381ac3a1dc39e7f99/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:31.064497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:31.075490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.177160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffbd32-e3a8-401a-a072-98fb2117d72b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:31.182069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:31.184785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.185240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.185942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.227185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:31.239794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab320f87-942a-41c2-88b5-6ce56e6cca26 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:31.264523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 699/1401] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:04:31.269996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.270408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.270763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.352761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] MFA Rules not processed for user `bb5ea3edc43743279902a002b8fe098c`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:31.359941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.360521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:31.361990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6877e0ef-4cb7-44c8-bba8-3464873e6481 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:31.403565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 703/1402] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:31.407415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.407915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.408054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.483120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] MFA Rules not processed for user `d50dfcf30325413381ac3a1dc39e7f99`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:31.485513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.578623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 700/1403] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:31.597870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:31.600882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.690429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 704/1404] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:31.706085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:31.706085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:31.752754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869b63e-0630-4923-a4c8-e9eb6c237bbd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:31.779838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:31.783242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.787122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.788393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.815612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:31.817485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:31.837303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 701/1405] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:31.892113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89833f4-7bd9-4742-9d59-9c4af704615d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:31.913391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:31.914514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:31.915033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:31.915632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:31.934608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:31.936569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d985094b-f701-4053-9eda-dfa7d59bf3de service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:31.979455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 705/1406] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:04:32.266052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:32.280701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2226bc3-28fd-4aec-86cd-a8d0c91b3f44 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:32.293616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:32.296299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:32.297046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:32.297340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:32.303320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:32.470826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:32.472620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:32.498453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 702/1407] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:32.612733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:32.909534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:33.618550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117a35-f53a-414c-8582-83f656076870 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:33.618550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117a35-f53a-414c-8582-83f656076870 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:33.618550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117a35-f53a-414c-8582-83f656076870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:33.733849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-82117a35-f53a-414c-8582-83f656076870 None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:33.735182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-82117a35-f53a-414c-8582-83f656076870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:33.769272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 706/1408] 10.222.0.16 () {58 vars in 1015 bytes} [Fri Oct 21 15:04:33 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:33.963223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:33.979435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f8ca8-37f7-4420-8b7e-7593e8ac2682 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:33.994284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:33.998647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:34.000439 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:34.005146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:34.029762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:34.043902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:34.051213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82117a35-f53a-414c-8582-83f656076870 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:34.078019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82117a35-f53a-414c-8582-83f656076870 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:34.107802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 703/1409] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:34.124887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:34.125570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:34.125854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:34.132826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] PATH_INFO: `/v3/users/7aad33b912b94cbf98948612bbe3e417` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:34.139444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aad33b912b94cbf98948612bbe3e417)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:34.146095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:34.575770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aad33b912b94cbf98948612bbe3e417'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:34.579307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aad33b912b94cbf98948612bbe3e417'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:34.615080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aad33b912b94cbf98948612bbe3e417'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:34.815543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:34.817404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:34.828938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aad33b912b94cbf98948612bbe3e417'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:34.829968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7a09dc0-297c-4ae8-b1e3-a7d281a06be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aad33b912b94cbf98948612bbe3e417'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:34.849416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f94dc0-fba9-4391-b2ef-c3f3687b3f0c service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:34.866816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:34.867737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:34.868423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:34.868988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] PATH_INFO: `/v3/projects/d4bcbf9e774d4c6fa12949219c837cfe` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:34.872138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4bcbf9e774d4c6fa12949219c837cfe)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:34.875510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 707/1410] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:34 2022] DELETE /identity/v3/users/7aad33b912b94cbf98948612bbe3e417 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:34.877718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:34.884490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:34.884979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:34.885435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:34.926374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:34.982104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:34.997399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:35.149730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4bcbf9e774d4c6fa12949219c837cfe', 'request_id': 'req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.240236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 708/1411] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:35.259483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4bcbf9e774d4c6fa12949219c837cfe', 'request_id': 'req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.270199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4bcbf9e774d4c6fa12949219c837cfe', 'request_id': 'req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.327484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] The token cache is being invalidate because project d4bcbf9e774d4c6fa12949219c837cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:35.327801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.330196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.334591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:35.336414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:35.342955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 704/1412] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:34 2022] DELETE /identity/v3/projects/d4bcbf9e774d4c6fa12949219c837cfe => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:35.347213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:35.350511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:35.506346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8dc87a-eaf0-408f-b5d0-1aabf1f9dc4f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:35.516635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:35.518939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:35.518939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:35.530584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] PATH_INFO: `/v3/users/fb45028753ee4a1bb10c8dd6ada5c8b7` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:35.533547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb45028753ee4a1bb10c8dd6ada5c8b7)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:35.536514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:35.618331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bf529c-35c3-49fe-8e05-8a04a1477d1d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:35.744652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:35.809674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:35.812664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:35.812664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:35.812664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:35.818322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb45028753ee4a1bb10c8dd6ada5c8b7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.819627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:35.846953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb45028753ee4a1bb10c8dd6ada5c8b7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.848922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb45028753ee4a1bb10c8dd6ada5c8b7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.858694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:35.858694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:35.870646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb45028753ee4a1bb10c8dd6ada5c8b7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.872765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb45028753ee4a1bb10c8dd6ada5c8b7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:35.939359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 709/1413] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:35.980484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 705/1414] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:35 2022] DELETE /identity/v3/users/fb45028753ee4a1bb10c8dd6ada5c8b7 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:37.117023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:37.159765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:37.161282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:37.199073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7daa0f-7ff1-4238-9461-b244c6177c81 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:37.403457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:37.406887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:37.409707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:37.409707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] PATH_INFO: `/v3/projects/5eb130530aa4458087264044c21e17ad` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:37.410896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eb130530aa4458087264044c21e17ad)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:37.419281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:37.693802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eb130530aa4458087264044c21e17ad', 'request_id': 'req-428af5ae-1851-4749-ac50-42d0672ebc9c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:37.754550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eb130530aa4458087264044c21e17ad', 'request_id': 'req-428af5ae-1851-4749-ac50-42d0672ebc9c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:37.771515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eb130530aa4458087264044c21e17ad', 'request_id': 'req-428af5ae-1851-4749-ac50-42d0672ebc9c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:37.809875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] The token cache is being invalidate because project 5eb130530aa4458087264044c21e17ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:37.812905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:37.812905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:37.817194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 710/1415] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:37 2022] DELETE /identity/v3/projects/5eb130530aa4458087264044c21e17ad => generated 0 bytes in 658 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:37.821249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:37.823934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:37.878501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20066f93-dc5c-43c9-9abd-e6eadb3ebaa4 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:38.122789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:38.123667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:38.150300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:38.151981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] PATH_INFO: `/v3/users/d50dfcf30325413381ac3a1dc39e7f99` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:38.165997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d50dfcf30325413381ac3a1dc39e7f99)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:38.165997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:38.484419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd50dfcf30325413381ac3a1dc39e7f99'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:38.540610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd50dfcf30325413381ac3a1dc39e7f99'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:38.541382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd50dfcf30325413381ac3a1dc39e7f99'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:38.553262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd50dfcf30325413381ac3a1dc39e7f99'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:38.554995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-45757a27-ca96-47f9-a800-7e58f36696f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd50dfcf30325413381ac3a1dc39e7f99'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:38.634682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 706/1416] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:37 2022] DELETE /identity/v3/users/d50dfcf30325413381ac3a1dc39e7f99 => generated 0 bytes in 811 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:38.734036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:38.735616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:38.756737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428af5ae-1851-4749-ac50-42d0672ebc9c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:38.919073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:38.919991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:38.920421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:38.920863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] PATH_INFO: `/v3/projects/e1dc7834d55848089160cf7752013f25` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:38.923106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1dc7834d55848089160cf7752013f25)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:38.927902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:38.933139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:39.064316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:39.138127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1dc7834d55848089160cf7752013f25', 'request_id': 'req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:39.206186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1dc7834d55848089160cf7752013f25', 'request_id': 'req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:39.214198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1dc7834d55848089160cf7752013f25', 'request_id': 'req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:39.241736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] The token cache is being invalidate because project e1dc7834d55848089160cf7752013f25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:39.242264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:39.243331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-dbb8ef3c-4c01-4ff3-94ba-b7ace27f5a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:39.245513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 711/1417] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:38 2022] DELETE /identity/v3/projects/e1dc7834d55848089160cf7752013f25 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:39.250432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa109a-d643-41da-81e7-182d0e8fbbc0 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.251377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa109a-d643-41da-81e7-182d0e8fbbc0 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.251988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa109a-d643-41da-81e7-182d0e8fbbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.299986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9efa109a-d643-41da-81e7-182d0e8fbbc0 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:39.301761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9efa109a-d643-41da-81e7-182d0e8fbbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:39.430930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53d668-6cf3-4c89-aa67-895ac1185e34 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.431226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53d668-6cf3-4c89-aa67-895ac1185e34 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.431511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53d668-6cf3-4c89-aa67-895ac1185e34 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.433757 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 712/1418] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:39.501991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.502566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.502956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.568401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 707/1419] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:39.583193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.584062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.584602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:39.585771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.587874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:39.691025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:39.700521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:39.741660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 708/1420] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:39.746094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:39.779231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34c591-898d-49ac-ab75-f28f2a4f0a01 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:39.782283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 713/1421] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:04:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:39.812512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:39.813293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.813664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.814199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.817516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:39.817516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:39.817516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:39.851806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:39.852063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 709/1422] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:39.857955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:39.863864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:39.873044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:39.880618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d905-a8e2-4a1b-8bf9-fc90da4682fb None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:39.903084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:39.905165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:39.905165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:39.905165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:39.905165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:39.923802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.023315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3efed2-2a3f-499b-af5f-70986800b59b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:40.059451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:40.082002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.082002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.082002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.082855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 714/1423] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:39 2022] POST /identity/v3/projects => generated 389 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:40.082855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:40.090192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:40.093694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.114413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 710/1424] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:04:40.297457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4146a7e-1ee5-4b9a-a249-d71cbf901ba5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:40.313813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:40.313813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.313813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.313813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.314162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:40.323924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.486404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 715/1425] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:40 2022] POST /identity/v3/users => generated 295 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:40.488985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:40.507640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336104d7-50f3-4253-a468-52c8d2f7cfdc service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:40.514350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:40.515485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.516075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.516767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.517788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:40.520316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.545246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:40.555385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 711/1426] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:40.559877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac29379-1d56-466c-82fe-5503f1769974 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:40.563710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:40.568396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:40.569515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.569515 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.573452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.573452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:40.573452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea37681-ecc5-4bb4-a6cc-6de05411e8d5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.606919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:40.616611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 716/1427] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:40.628375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef233d-d7d2-4181-9715-7b8a89c6c402 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:40.641053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:40.642025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.642672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.643681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] PATH_INFO: `/v3/projects/d8f82198e88d4f549ac8b82c7d971e8d/users/5b7a803ecc21444c8aa694259a1d92b4/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.694802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f82198e88d4f549ac8b82c7d971e8d, user_id=5b7a803ecc21444c8aa694259a1d92b4, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:40.699342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4df51-8322-4cca-a27e-55f5b990ddf3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:40.769909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298b26-ff4a-470d-8cc7-27f2d0c253c4 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.770747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298b26-ff4a-470d-8cc7-27f2d0c253c4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.771485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3298b26-ff4a-470d-8cc7-27f2d0c253c4 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.773712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 717/1428] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:04:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:40.788883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 712/1429] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:40 2022] PUT /identity/v3/projects/d8f82198e88d4f549ac8b82c7d971e8d/users/5b7a803ecc21444c8aa694259a1d92b4/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:40.801558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.802764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.803455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.805379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:40.806228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:40.806515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:40.851661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:40.855898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:40.926753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] MFA Rules not processed for user `5b7a803ecc21444c8aa694259a1d92b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:40.926753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:41.013358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:41.071117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 718/1430] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:41.074991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:41.078213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:41.144977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:41.171370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 713/1431] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:41.176949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:41.285983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a146bec-d2bf-468c-9b38-f071e5cc329f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:41.293317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:41.295529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:41.301931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:41.301931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] PATH_INFO: `/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:41.301931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fee6be7054a4e55b7f374e4c1c77ccc)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:41.303430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:41.380673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed637e5e-964e-451a-b14e-8eb3b205022e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:41.385809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:41.386826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:41.387192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:41.387587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:41.388249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:41.405589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:41.501385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fee6be7054a4e55b7f374e4c1c77ccc', 'request_id': 'req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:41.553693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 714/1432] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:41 2022] POST /identity/v3/projects => generated 389 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:41.554714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:41.575523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fee6be7054a4e55b7f374e4c1c77ccc', 'request_id': 'req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:41.610075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fee6be7054a4e55b7f374e4c1c77ccc', 'request_id': 'req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:41.643011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] The token cache is being invalidate because project 6fee6be7054a4e55b7f374e4c1c77ccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:41.643348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:41.644098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:41.649204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 719/1433] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:41 2022] DELETE /identity/v3/projects/6fee6be7054a4e55b7f374e4c1c77ccc => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:41.654181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:41.657363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:41.744513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e5bc9-ad20-4402-b441-2f80e048b8d5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:41.753259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:41.755081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:41.756343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:41.757121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:41.759245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:41.777073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:41.784412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:41.790431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9455b21-d668-4dda-a82b-a737725603a3 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:41.845352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:41.855113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5d38c5-3d4f-4a91-a0f8-4fc2191ea328 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:41.860317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 715/1434] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:41.861596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de85b-95aa-4401-b028-91d68e0f7357 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:41.861933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de85b-95aa-4401-b028-91d68e0f7357 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:41.862374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de85b-95aa-4401-b028-91d68e0f7357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:41.927995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-503de85b-95aa-4401-b028-91d68e0f7357 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:41.927995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-503de85b-95aa-4401-b028-91d68e0f7357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:42.162271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 716/1435] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:42.171080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.171365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.171645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.210536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:42.211832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:42.231285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 717/1436] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:42.235444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:42.258449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a624a21-8f77-47d0-871c-ee5d96f0ab96 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:42.270648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.272255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.272705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.273076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.273881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:42.277744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.282987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.303296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 718/1437] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:42.317650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:42.373395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd1ef-47fd-4a12-829b-35c5a0ad5751 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:42.396292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.398753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.399057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.400013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.407908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.416342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.417760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.418040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.418413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.419516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.419808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.422095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.583301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:42.593718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 720/1438] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:41 2022] POST /identity/v3/users => generated 301 bytes in 941 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:42.595665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:42.600539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 719/1439] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:42 2022] POST /identity/v3/projects => generated 385 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:42.602206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:42.614823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4731aa-4c7a-4132-9e40-74115c09cedc admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:42.770156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.770827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.771112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.771387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.774963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.774963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.799352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 721/1440] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:42.803633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:42.817233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f82be1-c05f-4cfc-9738-f740db2615d9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:42.832774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddbed15-fb5e-415f-b9a8-cb4e3ced99f8 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:42.837204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:42.843107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.843842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.844122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.844432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] PATH_INFO: `/v3/projects/937af0ef7f9942e5a93c063024be09b5/users/2fd7e7df226541c9a9b6ae7f0816eb1f/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.847547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:42.848441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:42.848925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:42.849203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:42.851214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.859917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.894254 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937af0ef7f9942e5a93c063024be09b5, user_id=2fd7e7df226541c9a9b6ae7f0816eb1f, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:42.898215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:42.986919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 720/1441] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:42 2022] POST /identity/v3/users => generated 293 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:42.988549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.022741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 722/1442] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:42 2022] PUT /identity/v3/projects/937af0ef7f9942e5a93c063024be09b5/users/2fd7e7df226541c9a9b6ae7f0816eb1f/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:43.027345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.032296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fae935-c3c1-493b-bfcf-3d1074946a16 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.127134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:43.171762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7831f-8ced-4468-b837-2e16df1f49d2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.177202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.178159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.178606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.180490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.181446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.184005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.196643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 723/1443] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:43.200834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.210127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:43.217068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ca811-a49d-45d6-9ff1-e31d9f85a452 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.222213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.223325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.223818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.224093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] PATH_INFO: `/v3/projects/937af0ef7f9942e5a93c063024be09b5/users/2fd7e7df226541c9a9b6ae7f0816eb1f/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.224242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.227742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=937af0ef7f9942e5a93c063024be09b5, user_id=2fd7e7df226541c9a9b6ae7f0816eb1f, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.227885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.228254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.228628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.230056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.231754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d15a69-d6a6-4ea5-b40f-ce8bec9f9406 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.236180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.245336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 724/1444] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:43 2022] PUT /identity/v3/projects/937af0ef7f9942e5a93c063024be09b5/users/2fd7e7df226541c9a9b6ae7f0816eb1f/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:43.259724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.260144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.260559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.295116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 721/1445] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:43.298969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.306184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] MFA Rules not processed for user `2fd7e7df226541c9a9b6ae7f0816eb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:43.308018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:43.322245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21473e90-cae5-45d3-b515-e4b1d78468b2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.429472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 725/1446] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:43.468486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.470552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:43.564344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.567445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.569045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.570473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.574043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.577764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.634911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 722/1447] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:43.641214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:43.645159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:43.669216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b13c0e7-b97d-48d4-baa2-4fd2ab8c997e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.681985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57904ad2-e40e-4482-be9e-b82bf2a0d849 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:43.684916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.687319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.690533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.691036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] PATH_INFO: `/v3/projects/8aefe386a9eb4333bbc26c79d058a9ee/users/5a188745a2284f0d8220003b768b4ef0/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.695002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:43.695886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.696116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.696456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.722735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.724802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c756a82-5a10-4197-a13a-6e5737d31f72 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.731015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aefe386a9eb4333bbc26c79d058a9ee, user_id=5a188745a2284f0d8220003b768b4ef0, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:43.736878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:43.737909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 726/1448] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:04:43.849739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 723/1449] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:43 2022] PUT /identity/v3/projects/8aefe386a9eb4333bbc26c79d058a9ee/users/5a188745a2284f0d8220003b768b4ef0/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:43.861570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:43.861859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:43.862181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:43.932780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] MFA Rules not processed for user `5a188745a2284f0d8220003b768b4ef0`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:43.935961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:44.134204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 727/1450] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:44.147798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:44.154358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:44.223730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:44.326768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475b9b8-9115-4a6c-b0f3-c3d89f580b05 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:44.335477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:44.336185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:44.336455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:44.336720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:44.368397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce0a1e6-1cc2-4afd-9a7f-cc03754c79f0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:44.371415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:44.376948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:44.379666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:44.380636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:44.380921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:44.381350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] PATH_INFO: `/v3/users/5b7a803ecc21444c8aa694259a1d92b4` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:44.383442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7a803ecc21444c8aa694259a1d92b4)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:44.388264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:44.418695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 724/1451] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:44.443895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:44.469482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d16b1b-efdd-4397-8a82-8923a9ed1156 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:44.476275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:44.476275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:44.482838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:44.483398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] PATH_INFO: `/v3/users/5a188745a2284f0d8220003b768b4ef0` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:44.485214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a188745a2284f0d8220003b768b4ef0)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:44.487637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:44.637646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7a803ecc21444c8aa694259a1d92b4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.639026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7a803ecc21444c8aa694259a1d92b4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.644158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:44.661084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a188745a2284f0d8220003b768b4ef0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.663527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7a803ecc21444c8aa694259a1d92b4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.676832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a188745a2284f0d8220003b768b4ef0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.677804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a188745a2284f0d8220003b768b4ef0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.692951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a188745a2284f0d8220003b768b4ef0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.693870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3a6f7397-8c68-48d3-8aca-82d6e3dd0be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a188745a2284f0d8220003b768b4ef0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.745461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7a803ecc21444c8aa694259a1d92b4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.746321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3b8f9591-2eba-4765-aa4e-630e054c3709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7a803ecc21444c8aa694259a1d92b4'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:44.752307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:44.766402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 728/1452] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:44 2022] DELETE /identity/v3/users/5b7a803ecc21444c8aa694259a1d92b4 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:44.780876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:44.781233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:44.781678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:44.811704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:44.813904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:44.824327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 725/1453] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:44 2022] DELETE /identity/v3/users/5a188745a2284f0d8220003b768b4ef0 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:44.833216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:44.833517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:44.834179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:44.909056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:44.912876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:44.941917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 729/1454] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:44.960169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:44.970549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:45.241646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 726/1455] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:45.242529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04302d5-4764-4171-9278-bbc95cb84135 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:45.383038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:45.450776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:45.450776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:45.450776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:45.450776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:45.460282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:45.479862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:45.519600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 730/1456] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:45.562622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788658ac-6943-4b23-89de-b7279db2b7f6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:45.571179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:45.571907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:45.572274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:45.574652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:45.588992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:45.600045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:45.631562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 727/1457] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:46.605134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:46.607378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:46.654171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9c59d-887a-455b-8933-5c8f9e972c20 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:46.686111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:46.687175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:46.687531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:46.687766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] PATH_INFO: `/v3/projects/d8f82198e88d4f549ac8b82c7d971e8d` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:46.689666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f82198e88d4f549ac8b82c7d971e8d)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:46.692048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:46.846070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:46.846070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:46.846070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c04855-fe04-4bbc-809a-bfa998802faf service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:46.848728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:46.848728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:46.848728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:46.848728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] PATH_INFO: `/v3/projects/8aefe386a9eb4333bbc26c79d058a9ee` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:46.849976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aefe386a9eb4333bbc26c79d058a9ee)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:46.849976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.242809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:47.244265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aefe386a9eb4333bbc26c79d058a9ee', 'request_id': 'req-eeeef348-9118-4332-9d73-9c9ab962cd81'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.246027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f82198e88d4f549ac8b82c7d971e8d', 'request_id': 'req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.246584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:47.342477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aefe386a9eb4333bbc26c79d058a9ee', 'request_id': 'req-eeeef348-9118-4332-9d73-9c9ab962cd81'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.350477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aefe386a9eb4333bbc26c79d058a9ee', 'request_id': 'req-eeeef348-9118-4332-9d73-9c9ab962cd81'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.362536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] The token cache is being invalidate because project 8aefe386a9eb4333bbc26c79d058a9ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:47.362676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.363415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-eeeef348-9118-4332-9d73-9c9ab962cd81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.364960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 728/1458] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:46 2022] DELETE /identity/v3/projects/8aefe386a9eb4333bbc26c79d058a9ee => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:47.369809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.370219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.370616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.397775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:47.399328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:47.440305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f82198e88d4f549ac8b82c7d971e8d', 'request_id': 'req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.446878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f82198e88d4f549ac8b82c7d971e8d', 'request_id': 'req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.471101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] The token cache is being invalidate because project d8f82198e88d4f549ac8b82c7d971e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:47.471395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.472356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-19ccd87c-e1bb-4a7d-95d9-05deebf90f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.476793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 729/1460] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:47.478724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.480733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:47.484817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 731/1460] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:46 2022] DELETE /identity/v3/projects/d8f82198e88d4f549ac8b82c7d971e8d => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:47.486600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.486921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.487191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.497861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995d973-00ec-424e-aa2a-f10a7142cfe1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:47.503234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:47.504320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.504320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.504670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] PATH_INFO: `/v3/users/2fd7e7df226541c9a9b6ae7f0816eb1f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.506145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fd7e7df226541c9a9b6ae7f0816eb1f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:47.508259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.521920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:47.523221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:47.532438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 732/1461] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:47.536739 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.547488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a32f2ad-6ef8-49cd-be8c-d51f1723ae53 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:47.552628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:47.553396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.553678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.553949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.554602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:47.554956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:47.556595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.569727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 733/1462] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:47.575351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.587446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37d81a-135b-4292-9759-3b16406e19d9 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:47.595773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:47.596532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.597232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.597513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.598858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:47.600884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.732723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 734/1463] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:47 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:47.736986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.801386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89262c1-1ab2-4ec7-a2ee-b6ac57693fbb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:47.806425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:47.807817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.808434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.808557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.809171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:47.811521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.831354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fd7e7df226541c9a9b6ae7f0816eb1f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.847561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fd7e7df226541c9a9b6ae7f0816eb1f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.847561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fd7e7df226541c9a9b6ae7f0816eb1f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.860537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fd7e7df226541c9a9b6ae7f0816eb1f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.861575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fd7e7df226541c9a9b6ae7f0816eb1f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:47.906047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 735/1464] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:47 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:47.910855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.910855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:47.923140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dd42-7374-4094-b60e-8cde3da49981 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:47.930279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:47.936391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:47.936391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:47.936571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 730/1465] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:47 2022] DELETE /identity/v3/users/2fd7e7df226541c9a9b6ae7f0816eb1f => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:47.936652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:47.936652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:47.936789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.936874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:47.937058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:47.950175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 736/1466] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:47.954675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:47.965999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704ce8e-7483-46aa-9f02-bad558a844cd admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.028923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f1c88-a82a-408c-a123-9dfe7d14db38 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.034598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.035419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.035860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.036201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] PATH_INFO: `/v3/users/bb5ea3edc43743279902a002b8fe098c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.037611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb5ea3edc43743279902a002b8fe098c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.040132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.046219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.047485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.047485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.047680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.048366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.050721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.066693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 737/1467] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:48.067767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:48.078411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a758e64-b5e2-4bbe-8cb8-1678af763b43 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.083035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.083686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.083962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.084235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] PATH_INFO: `/v3/projects/2fa115142e01451f9a327780d4f0ca8f/users/a488f48c5698496ab92983604ccfc257/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.099450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa115142e01451f9a327780d4f0ca8f, user_id=a488f48c5698496ab92983604ccfc257, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.102085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3452900-f9fe-41e0-a087-be8f4697a652 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.208162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 738/1468] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:48 2022] PUT /identity/v3/projects/2fa115142e01451f9a327780d4f0ca8f/users/a488f48c5698496ab92983604ccfc257/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:48.219321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.219899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.221046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.257352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb5ea3edc43743279902a002b8fe098c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.259346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] MFA Rules not processed for user `a488f48c5698496ab92983604ccfc257`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:48.261341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:48.271482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb5ea3edc43743279902a002b8fe098c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.272437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb5ea3edc43743279902a002b8fe098c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.282283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb5ea3edc43743279902a002b8fe098c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.283142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7fa5bc8a-eea2-4f7a-bd74-78e8b7fcff50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb5ea3edc43743279902a002b8fe098c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.332909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 739/1469] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:48.344889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:48.346327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:48.359936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 731/1470] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:47 2022] DELETE /identity/v3/users/bb5ea3edc43743279902a002b8fe098c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:48.361431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.361719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.362407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.387998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:48.389342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:48.421477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda959d-bbd1-4c73-87a5-b38bfdb23195 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.443979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:48.455247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 732/1471] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:48.467686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:48.470415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:48.498659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.498659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.498659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.498659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.562606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7907e45d-b206-4962-a5fd-1c8bcc8cd6c3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.568037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.568674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.568937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.569214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.571426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.575413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.583372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.584814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c8507-566b-40ff-abe8-f04e6f117605 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.586669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 740/1472] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:48.595314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 733/1473] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:48.663313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:48.665168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:48.692845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d471e275-fbb2-4f9c-bb6f-472a1095bcc9 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:48.707425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:48.708367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:48.708808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:48.709282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] PATH_INFO: `/v3/projects/937af0ef7f9942e5a93c063024be09b5` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:48.711032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] RBAC: Authorizing `identity:delete_project(project_id=937af0ef7f9942e5a93c063024be09b5)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:48.716475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:48.853533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '937af0ef7f9942e5a93c063024be09b5', 'request_id': 'req-02a003d6-6348-4802-9676-65826b12f833'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:48.871268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:48.994011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '937af0ef7f9942e5a93c063024be09b5', 'request_id': 'req-02a003d6-6348-4802-9676-65826b12f833'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.000214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '937af0ef7f9942e5a93c063024be09b5', 'request_id': 'req-02a003d6-6348-4802-9676-65826b12f833'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.016829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] The token cache is being invalidate because project 937af0ef7f9942e5a93c063024be09b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:49.016829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.016829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.016829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 741/1474] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:48 2022] DELETE /identity/v3/projects/937af0ef7f9942e5a93c063024be09b5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:49.019009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052aee-47d6-45b5-8743-d3cd92028b1d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.019347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052aee-47d6-45b5-8743-d3cd92028b1d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.019653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7052aee-47d6-45b5-8743-d3cd92028b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.039934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c7052aee-47d6-45b5-8743-d3cd92028b1d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:49.041146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c7052aee-47d6-45b5-8743-d3cd92028b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.092713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.094350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.107044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 734/1475] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:49.115937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.116220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.116667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.137046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:49.139259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.148560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 735/1476] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:49.153714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.163789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc156d-a07a-4730-b8f6-1ac4ab78cf74 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.169950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.170720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.171063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.171402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.172174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:49.172538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.174745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.180364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a003d6-6348-4802-9676-65826b12f833 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.187470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.189235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.189235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.189235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] PATH_INFO: `/v3/projects/d5c6293e30b94963ada751aba24a570f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.190908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5c6293e30b94963ada751aba24a570f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.192968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 736/1477] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:04:49.193090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.193857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.204462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6234c-668c-4409-8a98-fdb44440f2ce admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.208970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.209665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.209948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.210218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.210878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.212769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.305622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5c6293e30b94963ada751aba24a570f', 'request_id': 'req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.340394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 737/1478] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/projects => generated 385 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:49.343779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.382350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5c6293e30b94963ada751aba24a570f', 'request_id': 'req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.394350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5c6293e30b94963ada751aba24a570f', 'request_id': 'req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.406869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba35228-8e1a-4e07-8600-bf0107f643cb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.407338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] The token cache is being invalidate because project d5c6293e30b94963ada751aba24a570f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:49.407635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.408283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5ddbfad0-21e3-4ee7-b2df-893be7e7fb88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:49.410049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 742/1479] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:49 2022] DELETE /identity/v3/projects/d5c6293e30b94963ada751aba24a570f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:49.411556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.412170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.412451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.412726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.413358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.414994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f9ae-f16d-4bbb-aae6-98a4ca16f931 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.415286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f9ae-f16d-4bbb-aae6-98a4ca16f931 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.415559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56f9ae-f16d-4bbb-aae6-98a4ca16f931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.415693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.435557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2d56f9ae-f16d-4bbb-aae6-98a4ca16f931 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:49.436781 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2d56f9ae-f16d-4bbb-aae6-98a4ca16f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.444691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 743/1480] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:49.453301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.453547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.453819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.474102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:49.475359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.484975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 744/1481] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:49.489023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.498132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e8451b-6906-4a70-ab71-f6fb1fecee35 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.502750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.503342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.503623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.503899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.504538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:49.504840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.506464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.518597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 738/1482] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:49.520991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 745/1483] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:49.526617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.529069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.531678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.543974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f7e3b7-ab55-4893-ae61-58a863885482 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.544305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26af91-bbe0-463c-8b80-dea3befc54ac admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.549752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.550205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.550539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.550890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.550990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.551169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.551283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.551435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.551940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.552098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.553653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.554940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.575886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 739/1484] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:49.580023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.592504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018ecdd-1885-44da-ae59-ac733f610cdf admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.598412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.599177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.599467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.599767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.600421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.602096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.614163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 740/1485] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:49.618168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.628409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8603427-acc7-41ba-929a-8204e220d78a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.633984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.633984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.633984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.634226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] PATH_INFO: `/v3/projects/28163e94d58c49ec8c0d13032d4f4a39/users/67a69399b1a6477c8ec9e9b24ba924ed/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.636975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 746/1486] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:49.640891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.655621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28163e94d58c49ec8c0d13032d4f4a39, user_id=67a69399b1a6477c8ec9e9b24ba924ed, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.659146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e24c0d-a9f1-4bc9-90b5-92af07fcfb61 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.706584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbf6ce-9921-4769-981a-3b499e9cb7e3 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.711151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.711769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.712055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.712332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.712996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.715056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.738685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 741/1487] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:49 2022] PUT /identity/v3/projects/28163e94d58c49ec8c0d13032d4f4a39/users/67a69399b1a6477c8ec9e9b24ba924ed/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:49.748895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.749186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.749503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.782366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] MFA Rules not processed for user `67a69399b1a6477c8ec9e9b24ba924ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:49.788942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:49.806467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 747/1488] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:49.811488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.825568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a41ba1-649a-4692-bfc4-e908d21f82c2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.862076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 742/1489] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:49.866296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.875434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:49.891561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.892194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.892841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.892841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.893485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.895139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.907905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 748/1490] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:49.912582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.924939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcc5373-cdf7-4227-86de-cbacdc50abc5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.931388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.932846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.933209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.933557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.934385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.936301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.951197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 749/1491] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:49.955345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:49.961385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3185d0c7-89d4-4e8f-a333-5bde544b2dd1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.966842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.967461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.967736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.968006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.968284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65405158-7dbc-417f-8165-0ece6a105c92 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:49.968635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.970658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:49.973710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:49.973710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:49.973923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:49.974103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] PATH_INFO: `/v3/projects/88041d93af3c482aa7520f37dec45bb8/users/22644a5e841f4a44943483a74dabeef3/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:49.990272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88041d93af3c482aa7520f37dec45bb8, user_id=22644a5e841f4a44943483a74dabeef3, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:49.992990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752407bc-7ee4-48fe-bf09-acf244edbc76 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.069217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 743/1492] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:49 2022] POST /identity/v3/projects => generated 387 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:50.073941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.105770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 750/1493] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:49 2022] PUT /identity/v3/projects/88041d93af3c482aa7520f37dec45bb8/users/22644a5e841f4a44943483a74dabeef3/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:50.117075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.118025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.118302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.135560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0d7dfc-7753-473b-998f-c34de84bde90 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.146886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] MFA Rules not processed for user `22644a5e841f4a44943483a74dabeef3`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:50.148143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:50.191343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.191963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.192261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.192544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.193177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.195133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.215312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 751/1494] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:50.219848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.222822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:50.230272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc02ca9-0966-4e16-bdb6-934b8ed27cc5 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.235189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.235774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.236054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.236327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.236948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.238905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.282970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 744/1495] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/users => generated 300 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:50.284139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.295364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ac441-830f-444f-a59d-c46517785f85 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.300180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.300784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.301082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.301357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.301980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.303590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.315424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 745/1496] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:50.318358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 752/1497] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:50.319416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.322341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.331816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df596c27-5136-452d-b95b-3c2ea9dedbd3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.404388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.405354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.405802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.406240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] PATH_INFO: `/v3/projects/88162283dbfb4f0189918430ed845085/users/1dd9e7780fec4d3f895bf7a88dea0f6f/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.408378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5966e-79d9-4f73-9819-16a4cfeb1e60 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.413374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.413986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.414520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.414903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.415544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.418179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.423879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88162283dbfb4f0189918430ed845085, user_id=1dd9e7780fec4d3f895bf7a88dea0f6f, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.426713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.434775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:50.531678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 753/1498] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:50.535415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.545672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f5b8-29b3-49e8-b00b-fec90c4823af admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.550405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.551021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.551824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.551824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.552256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.553867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.563821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 754/1499] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:50.567570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.577034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0c184-4fe7-46c6-a9c7-ce96840bc7ee admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.581702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.582281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.582563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.582904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.583524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.585072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.600556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 755/1500] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:50.600556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.610734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9243c-4b54-4077-aff7-5d244c70762e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.615221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.615831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.616104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.616373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] PATH_INFO: `/v3/projects/dc5ed13ed5144016b98aafb38e76ae8c/users/3d927174d3d5401b814f57f8b41b4506/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.629182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 746/1501] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:50 2022] PUT /identity/v3/projects/88162283dbfb4f0189918430ed845085/users/1dd9e7780fec4d3f895bf7a88dea0f6f/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:50.631096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc5ed13ed5144016b98aafb38e76ae8c, user_id=3d927174d3d5401b814f57f8b41b4506, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.633010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.633841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f3a8e3-f801-4a73-b7b0-416db9b5f210 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.694417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c43f87-2349-452f-83c5-ec6144ebde1f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.699433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.700041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.700329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.700597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.701310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.702943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.712522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 747/1502] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:50.717067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.720544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 756/1503] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:50 2022] PUT /identity/v3/projects/dc5ed13ed5144016b98aafb38e76ae8c/users/3d927174d3d5401b814f57f8b41b4506/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:50.726556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc8296-3eda-41ee-b122-4ccc85e95b72 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.731262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.731683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.731795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.759758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] MFA Rules not processed for user `3d927174d3d5401b814f57f8b41b4506`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:50.761565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:50.785918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.786718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.787139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.787443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] PATH_INFO: `/v3/projects/88162283dbfb4f0189918430ed845085/users/1dd9e7780fec4d3f895bf7a88dea0f6f/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.789729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88162283dbfb4f0189918430ed845085, user_id=1dd9e7780fec4d3f895bf7a88dea0f6f, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.793330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82741fe9-6e49-4b5e-a152-a7af00d997c1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.806160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 748/1504] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:50 2022] PUT /identity/v3/projects/88162283dbfb4f0189918430ed845085/users/1dd9e7780fec4d3f895bf7a88dea0f6f/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:50.817382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.817672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.817951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.847345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 757/1505] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:50.851965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.855379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] MFA Rules not processed for user `1dd9e7780fec4d3f895bf7a88dea0f6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:50.856838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:50.864943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e86054-0c8d-4d7c-8148-44e60635bffe None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:50.938777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 749/1506] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:50.940827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:50.941611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:50.941969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:50.943877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:50.943877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:50.945816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:50.967736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:50.969182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:51.029104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec4756-dbd9-4a76-b7fd-8b5f3d025876 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.047030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.047030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.047030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.047030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.059852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.061349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.074763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 750/1507] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:51.078291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 758/1508] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:50 2022] POST /identity/v3/projects => generated 379 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:51.082234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.094180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e498d77-bf93-4da0-975e-d2a19797a410 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.247602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:51.279102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.279715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.279816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:51.279987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.280366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.280888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.282831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.452457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 751/1509] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:51 2022] POST /identity/v3/users => generated 296 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:51.456328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.470595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a3079-d289-4286-bebd-2e5bb513d4ea admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.479825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.480613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.480894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.481164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.481815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.483531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.496861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 759/1510] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:51.500905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.512179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b77b6-c349-47f9-b5b7-01d12a2b4ea0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.517303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.518012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.518288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.521007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] PATH_INFO: `/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675/users/e779199182754ea3a7271ead8ce5278c/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.538084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf141afb8bd4299bd3b55f88b2e5675, user_id=e779199182754ea3a7271ead8ce5278c, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.541086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.582260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:51.662983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 752/1511] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:51 2022] PUT /identity/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675/users/e779199182754ea3a7271ead8ce5278c/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:51.667123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.677109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d5ff67-210e-4ff9-be7d-9cc970a62647 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.681905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:51.736352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.736352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.736352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.736352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.736783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.738493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa63ddd1-1d43-48f8-a58a-72aea980c584 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.749130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 760/1512] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:51.754168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.766035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5f226b-c079-4135-8f36-f5c847b1cd03 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.771191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.771853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.772271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.772431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] PATH_INFO: `/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675/users/e779199182754ea3a7271ead8ce5278c/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.774422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf141afb8bd4299bd3b55f88b2e5675, user_id=e779199182754ea3a7271ead8ce5278c, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.778036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:51.788998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 753/1513] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:51 2022] PUT /identity/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675/users/e779199182754ea3a7271ead8ce5278c/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:51.799745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.800053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.800327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.830070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] MFA Rules not processed for user `e779199182754ea3a7271ead8ce5278c`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:51.831322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:51.909562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 761/1514] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:51.921713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:51.954311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:51.980490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a06cb5-2958-4cc6-bc44-e96bc81556a0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:51.984402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:51.985223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:51.985511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:51.985787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:51.995837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:51.997377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:52.007006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 754/1515] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:52.172108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:52.257789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:52.257789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8494e-d0ec-4e27-8447-24fe56f1678b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:52.257789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:52.257789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:52.258577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:52.258577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:52.320650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:52.322132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71abc47f-b783-4b85-9d2e-a37853e5249a service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:52.331828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 762/1516] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:53.173406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:53.249231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c06b89c-0245-4bf9-8f94-b2a269fdddf7 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:53.254517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:53.255238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:53.255472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:53.255802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] PATH_INFO: `/v3/users/67a69399b1a6477c8ec9e9b24ba924ed` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:53.256975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67a69399b1a6477c8ec9e9b24ba924ed)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:53.258989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:53.319758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:53.415236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67a69399b1a6477c8ec9e9b24ba924ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:53.430717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67a69399b1a6477c8ec9e9b24ba924ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:53.431845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67a69399b1a6477c8ec9e9b24ba924ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:53.440618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67a69399b1a6477c8ec9e9b24ba924ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:53.441322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67a69399b1a6477c8ec9e9b24ba924ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:53.502284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 755/1517] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:53 2022] DELETE /identity/v3/users/67a69399b1a6477c8ec9e9b24ba924ed => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:53.507028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:53.507552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:53.508191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:53.529250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:53.530893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:53.600660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 763/1518] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:53.610713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:53.612211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:53.669669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b4c620-c352-4e9c-bec8-897d982c6500 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:53.673570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:53.674182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:53.674551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:53.674853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:53.684730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:53.686195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:53.695676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 756/1519] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:04:53.708778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:54.223844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:54.225494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:54.240693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea336a9f-3dff-4a34-8930-888cee142e68 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:54.246875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:54.247521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:54.247830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:54.248133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] PATH_INFO: `/v3/projects/28163e94d58c49ec8c0d13032d4f4a39` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:54.249404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28163e94d58c49ec8c0d13032d4f4a39)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:54.251361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:54.404751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:54.417818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28163e94d58c49ec8c0d13032d4f4a39', 'request_id': 'req-7914324b-9772-4e0a-b773-8a0070eb1970'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.496664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28163e94d58c49ec8c0d13032d4f4a39', 'request_id': 'req-7914324b-9772-4e0a-b773-8a0070eb1970'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.502828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28163e94d58c49ec8c0d13032d4f4a39', 'request_id': 'req-7914324b-9772-4e0a-b773-8a0070eb1970'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.514778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] The token cache is being invalidate because project 28163e94d58c49ec8c0d13032d4f4a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:54.515316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.515752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.517248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 764/1520] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:54 2022] DELETE /identity/v3/projects/28163e94d58c49ec8c0d13032d4f4a39 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:54.526590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:54.531542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820f036-231b-4902-91a1-df044d070ab4 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:54.600042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:54.600770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:54.601010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:54.601277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] PATH_INFO: `/v3/users/1dd9e7780fec4d3f895bf7a88dea0f6f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:54.602393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dd9e7780fec4d3f895bf7a88dea0f6f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:54.605047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:54.797193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dd9e7780fec4d3f895bf7a88dea0f6f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.827596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dd9e7780fec4d3f895bf7a88dea0f6f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.829373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dd9e7780fec4d3f895bf7a88dea0f6f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.849209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dd9e7780fec4d3f895bf7a88dea0f6f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.850108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f4ad472f-e446-4160-aad4-0140812c6a43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dd9e7780fec4d3f895bf7a88dea0f6f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:54.854672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:54.949381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 757/1521] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:04:54 2022] DELETE /identity/v3/users/1dd9e7780fec4d3f895bf7a88dea0f6f => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:55.384985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:55.386799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:55.401059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7914324b-9772-4e0a-b773-8a0070eb1970 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:55.462780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:55.463611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:55.463981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:55.464253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] PATH_INFO: `/v3/projects/88162283dbfb4f0189918430ed845085` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:55.465384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88162283dbfb4f0189918430ed845085)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:55.471018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:55.570489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88162283dbfb4f0189918430ed845085', 'request_id': 'req-0ddcceb6-3691-48c4-9852-620273549f1c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:55.638925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88162283dbfb4f0189918430ed845085', 'request_id': 'req-0ddcceb6-3691-48c4-9852-620273549f1c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:55.644570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88162283dbfb4f0189918430ed845085', 'request_id': 'req-0ddcceb6-3691-48c4-9852-620273549f1c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:55.655938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] The token cache is being invalidate because project 88162283dbfb4f0189918430ed845085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:04:55.656226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:55.656899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0ddcceb6-3691-48c4-9852-620273549f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:04:55.658368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 765/1522] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:04:55 2022] DELETE /identity/v3/projects/88162283dbfb4f0189918430ed845085 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:55.663275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:55.663868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:55.663868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:55.684530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:55.685767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:55.743478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 758/1523] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:55.750950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:55.751239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:55.751512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:55.770840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:55.772028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:55.780819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 766/1524] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:55.785168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:55.795691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f3837-b8fe-4baa-ace2-1dd6dd9a1890 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:55.800184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:55.800778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:55.801057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:55.801337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:55.801976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:04:55.802278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:55.803889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:55.815360 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 759/1525] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:04:55.819497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:55.829247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc7ad97-ebe9-4736-bbaa-ca6703c3d240 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:55.834045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:55.834585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:55.834888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:55.835160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:55.835803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:55.837682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:55.959722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 767/1526] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:55 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:55.961043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:55.980445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589399d5-62a2-417f-ab38-32f218f05e68 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.039959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.040477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.040757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.041029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.042286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.044260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.158008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 760/1527] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:55 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:56.159204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.170320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfef02-45bd-428e-863c-c6655f440a92 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.175526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.176173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.176551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.176825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.177457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.179107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.189709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 768/1528] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:56.193849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.205180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd511b-0b92-4652-a499-9809909512b8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.209991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.210641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.210931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.211200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.211834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.214986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b8862-8a67-4da2-a42b-d8b71ba74101 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.244432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 761/1529] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:56.248748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.258950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c923bfb-4448-4c9e-a297-62a2111cd8a1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.264355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.265123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.265471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.265819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] PATH_INFO: `/v3/projects/e77ae4c4746b49ae8da4fcd08fb6d292/users/033602599586468ca8cb69efdf3dd725/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.269394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:56.281624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77ae4c4746b49ae8da4fcd08fb6d292, user_id=033602599586468ca8cb69efdf3dd725, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.284426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.363749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 769/1530] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:56 2022] PUT /identity/v3/projects/e77ae4c4746b49ae8da4fcd08fb6d292/users/033602599586468ca8cb69efdf3dd725/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:56.365464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.365765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.366041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.395063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] MFA Rules not processed for user `033602599586468ca8cb69efdf3dd725`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:56.396265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:56.443079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:56.462419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 762/1531] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:56.467070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.529025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468cd349-d30d-4737-ba82-d5d46b7e938b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.533625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.534218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.534498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.534805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.535442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.537342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.628961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 770/1532] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:56 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:56.633609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.644408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169efc1b-9efc-4258-b4c6-97bd8e96274a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.701501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.702118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.702369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.702654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.703301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.705195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.794250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 763/1533] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:56 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:56.798268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.808747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9c939-a9dd-41fc-b1e4-4de32da04f70 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.813299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.813896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.814177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.814448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.815119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.823132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.828758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 771/1534] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:56.832764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.843930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c46f53-866a-4490-a38f-ce2442cfc1c9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.848482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.849069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.849348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.849613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.850242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.851855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c311c849-dc2c-4419-8ec9-94e367affc7f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.862093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 764/1535] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:56.866227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:56.876183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91ce7b-9647-49e8-bcdf-ee7f001ac249 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:56.880865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:56.881453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:56.881734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:56.882003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] PATH_INFO: `/v3/projects/59a1f04152c849d7a7963b0fd29b2f4d/users/5e3efadd3f714b3fab4f58c7a8f637b8/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:56.896953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a1f04152c849d7a7963b0fd29b2f4d, user_id=5e3efadd3f714b3fab4f58c7a8f637b8, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:56.899686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:56.988133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 772/1536] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:56 2022] PUT /identity/v3/projects/59a1f04152c849d7a7963b0fd29b2f4d/users/5e3efadd3f714b3fab4f58c7a8f637b8/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:56.999899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.000179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.000451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.029230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] MFA Rules not processed for user `5e3efadd3f714b3fab4f58c7a8f637b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:57.030423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:57.105984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 765/1537] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:04:57.107071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.183329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ba4d-e256-4507-b167-d8e8f844143c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.192632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.193318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.194013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.194013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.195381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.200456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3811637e-b4a4-459c-a883-729cc507fa6a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.336177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 773/1538] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:04:57 2022] POST /identity/v3/projects => generated 377 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:04:57.337563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.352789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79535144-61fa-4fa9-9bfc-2a01ffce08fe None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.386867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32153-5710-4edf-a099-e7892f73eacd None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.387157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32153-5710-4edf-a099-e7892f73eacd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.387435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c32153-5710-4edf-a099-e7892f73eacd None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.389572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 774/1539] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:04:57.413092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.413382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.413669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.457634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:57.469415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.469756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.470043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.470323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.471100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.471234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:57.474397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.474934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:57.497177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:04:57.575912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 775/1540] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:57.624811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 766/1541] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:04:57 2022] POST /identity/v3/users => generated 295 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:04:57.629502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.638725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.640522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:57.643109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc22803-5a2b-4823-88ff-87a233739df8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.650564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.651889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.652171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.652445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.653215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.654969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.665327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d8ffaa-4ba8-44d5-b794-d31230dd6cfd admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.665906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 776/1542] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:04:57.670170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.671826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.672566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.672863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.673143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.683212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.685860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.686078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a640b376-dfbe-4b7a-9650-3c37f58a065a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.695336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.695336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.695336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.695336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] PATH_INFO: `/v3/projects/eb176a2a41614c30b894e2787bfec4f7/users/c7ee9ca87ed74eb583885a1d0904e63e/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.698127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 767/1543] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:04:57.710832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb176a2a41614c30b894e2787bfec4f7, user_id=c7ee9ca87ed74eb583885a1d0904e63e, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.714148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.823612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 777/1544] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:57 2022] PUT /identity/v3/projects/eb176a2a41614c30b894e2787bfec4f7/users/c7ee9ca87ed74eb583885a1d0904e63e/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:04:57.827666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.837923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4198fef-c984-4246-ac86-8f710672e789 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.895569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.896168 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.896445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.896712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.897339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.898994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981910fb-10a8-42da-bb5c-d56b2ff2f8bf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.909305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 768/1545] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:04:57.913701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:57.924528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb8f677-38fa-44f6-bfc9-1e168417aba4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:57.929160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:57.929830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.930118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.930414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] PATH_INFO: `/v3/projects/eb176a2a41614c30b894e2787bfec4f7/users/c7ee9ca87ed74eb583885a1d0904e63e/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:57.933014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb176a2a41614c30b894e2787bfec4f7, user_id=c7ee9ca87ed74eb583885a1d0904e63e, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:57.935911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:57.958461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 778/1546] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:04:57 2022] PUT /identity/v3/projects/eb176a2a41614c30b894e2787bfec4f7/users/c7ee9ca87ed74eb583885a1d0904e63e/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:04:57.960053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:57.960411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:57.960761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:58.001887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] MFA Rules not processed for user `c7ee9ca87ed74eb583885a1d0904e63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:04:58.003449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:58.117779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 769/1547] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:04:58.151662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:04:58.153587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:04:58.234801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a7462-150c-4f1b-9c93-ae0ec52a933d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:04:58.239222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:04:58.239901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:04:58.240267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:04:58.240773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:04:58.299660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:04:58.301103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:04:58.310825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 779/1548] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:04:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:02.716638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:02.727202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee650-52c2-493d-8e65-2a27dc016a21 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:02.732161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:02.732795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:02.733077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:02.733353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:02.808749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:02.819759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:02.822593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4b8839-964b-48f8-9948-c5c2e48b19e1 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:02.856018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 770/1549] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:02.942249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:02.943940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:02.966071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:03.023030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520d3e0-7f3a-4158-9fa7-92e9358c0757 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:03.031117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:03.031893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:03.032171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:03.032444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] PATH_INFO: `/v3/users/22644a5e841f4a44943483a74dabeef3` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:03.033648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22644a5e841f4a44943483a74dabeef3)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:03.036040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:03.270654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22644a5e841f4a44943483a74dabeef3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:03.271386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22644a5e841f4a44943483a74dabeef3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:03.290102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22644a5e841f4a44943483a74dabeef3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:03.388323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22644a5e841f4a44943483a74dabeef3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:03.388958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22644a5e841f4a44943483a74dabeef3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:03.402382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 780/1550] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:02 2022] DELETE /identity/v3/users/22644a5e841f4a44943483a74dabeef3 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:03.407611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:03.407944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:03.408253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:03.429565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:03.430813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:03.493272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 771/1551] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:03.500375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:03.501831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:03.565500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3ab1c3-825e-4894-90f1-6c1b5cb51249 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:03.570045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:03.570910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:03.571192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:03.571499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:03.588541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:03.590905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:03.602033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 781/1552] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:04.275663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:04.277289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:04.293398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46d6099-15d3-4a07-9317-6c7b8e69a2bd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:04.299086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:04.299735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:04.300051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:04.300276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] PATH_INFO: `/v3/projects/88041d93af3c482aa7520f37dec45bb8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:04.301562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=88041d93af3c482aa7520f37dec45bb8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:04.303526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:04.423508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88041d93af3c482aa7520f37dec45bb8', 'request_id': 'req-deef1a70-9929-4b27-b4dc-201fe3176c2d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.507907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88041d93af3c482aa7520f37dec45bb8', 'request_id': 'req-deef1a70-9929-4b27-b4dc-201fe3176c2d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.516002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88041d93af3c482aa7520f37dec45bb8', 'request_id': 'req-deef1a70-9929-4b27-b4dc-201fe3176c2d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.527624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] The token cache is being invalidate because project 88041d93af3c482aa7520f37dec45bb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:04.527787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.528414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.529986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 772/1553] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:04 2022] DELETE /identity/v3/projects/88041d93af3c482aa7520f37dec45bb8 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:04.534123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:04.544686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8cfb3c-ad7d-4802-aa50-7d163db85893 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:04.586324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:04.605936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:04.606667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:04.608884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:04.608884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] PATH_INFO: `/v3/users/3d927174d3d5401b814f57f8b41b4506` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:04.608884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d927174d3d5401b814f57f8b41b4506)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:04.610207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:04.863321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d927174d3d5401b814f57f8b41b4506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.863321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d927174d3d5401b814f57f8b41b4506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.886353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d927174d3d5401b814f57f8b41b4506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.966372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d927174d3d5401b814f57f8b41b4506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:04.967716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d927174d3d5401b814f57f8b41b4506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.001178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 782/1554] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:04 2022] DELETE /identity/v3/users/3d927174d3d5401b814f57f8b41b4506 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:05.049328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:05.050919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:05.144049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef1a70-9929-4b27-b4dc-201fe3176c2d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:05.176028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:05.195809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:05.196464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:05.196732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:05.197002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:05.199678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:05.304725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:05.306167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:05.319683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 773/1555] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:05.585214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:05.586768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:05.683235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9c110-13da-44b4-8b0d-d63113dc23bd admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:05.688066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:05.688700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:05.688979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:05.689334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] PATH_INFO: `/v3/projects/dc5ed13ed5144016b98aafb38e76ae8c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:05.696881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc5ed13ed5144016b98aafb38e76ae8c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:05.696881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:05.829520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc5ed13ed5144016b98aafb38e76ae8c', 'request_id': 'req-31c813aa-f92f-4164-97f5-15466580c195'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.925083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc5ed13ed5144016b98aafb38e76ae8c', 'request_id': 'req-31c813aa-f92f-4164-97f5-15466580c195'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.929763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:05.934084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc5ed13ed5144016b98aafb38e76ae8c', 'request_id': 'req-31c813aa-f92f-4164-97f5-15466580c195'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.947081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] The token cache is being invalidate because project dc5ed13ed5144016b98aafb38e76ae8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:05.947403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.948092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:05.951045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 783/1556] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:05 2022] DELETE /identity/v3/projects/dc5ed13ed5144016b98aafb38e76ae8c => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:05.955381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:05.968948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918521aa-d410-4c35-a32d-bd0d9f44fe65 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:06.031719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:06.032493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:06.032852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:06.033197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] PATH_INFO: `/v3/users/e779199182754ea3a7271ead8ce5278c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:06.034891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e779199182754ea3a7271ead8ce5278c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:06.037193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:06.304107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e779199182754ea3a7271ead8ce5278c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:06.318674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e779199182754ea3a7271ead8ce5278c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:06.319393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e779199182754ea3a7271ead8ce5278c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:06.327044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e779199182754ea3a7271ead8ce5278c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:06.327694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5684257d-c6f4-460a-a3aa-e486f235071f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e779199182754ea3a7271ead8ce5278c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:06.405488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:06.412635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 774/1557] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:05 2022] DELETE /identity/v3/users/e779199182754ea3a7271ead8ce5278c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:07.107975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:07.109462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:07.128749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c813aa-f92f-4164-97f5-15466580c195 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:07.196448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:07.197330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.197619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.197897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] PATH_INFO: `/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.215537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf141afb8bd4299bd3b55f88b2e5675)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:07.217539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:07.393998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf141afb8bd4299bd3b55f88b2e5675', 'request_id': 'req-42b3f59a-2266-4d0d-a16f-ff070f6a304d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:07.500842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf141afb8bd4299bd3b55f88b2e5675', 'request_id': 'req-42b3f59a-2266-4d0d-a16f-ff070f6a304d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:07.510128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf141afb8bd4299bd3b55f88b2e5675', 'request_id': 'req-42b3f59a-2266-4d0d-a16f-ff070f6a304d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:07.523628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] The token cache is being invalidate because project 2bf141afb8bd4299bd3b55f88b2e5675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:07.528112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:07.528112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-42b3f59a-2266-4d0d-a16f-ff070f6a304d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:07.528112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 784/1558] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:07 2022] DELETE /identity/v3/projects/2bf141afb8bd4299bd3b55f88b2e5675 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:07.532106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.532601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.532680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.555767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:07.557051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:07.655811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 775/1559] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:07.665888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.666182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.666462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.690359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:07.692744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:07.704698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 785/1560] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:07.706176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:07.718571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f6bc40-df8f-4c89-8dff-814c8629bec8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:07.723456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:07.724149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.724484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.724843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.725560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:07.726454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:07.728159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:07.741347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 776/1561] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:07.747445 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:07.757452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a925083-f0a7-4047-9c8a-829118fd89e5 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:07.762439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:07.763092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.763400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.763680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.764321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:07.766232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:07.896135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 786/1562] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:07 2022] POST /identity/v3/projects => generated 371 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:07.897373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:07.908185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edf43fa-a624-403d-bbe9-cc2b11893845 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:07.973349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:07.974198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:07.974656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:07.975221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:07.975868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:07.977814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:08.116838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 777/1563] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:07 2022] POST /identity/v3/users => generated 286 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:08.118250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:08.128792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88b9f2-df99-442f-b01e-2d11aaeb34ca admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:08.134402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:08.135042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:08.138357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:08.138357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:08.138357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:08.138357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:08.152459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 787/1564] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:08.153612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:08.163990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bec072-1446-4fa4-8502-3812f3a40d07 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:08.168958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:08.169560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:08.169837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:08.170105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:08.170796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:08.172444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4a393-4289-42fc-8685-003202c8ab39 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:08.183403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 778/1565] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:08.187769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:08.198002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782f3a7-bc52-44c1-b58e-81c63dfb1a70 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:08.203372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:08.204093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:08.204371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:08.204639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] PATH_INFO: `/v3/projects/ea902dc770e449bd9e53b96f753176bf/users/c2ec23eb23ae4c1e8ddc37e73d759733/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:08.221089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea902dc770e449bd9e53b96f753176bf, user_id=c2ec23eb23ae4c1e8ddc37e73d759733, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:08.223955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:08.322601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 788/1566] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:08 2022] PUT /identity/v3/projects/ea902dc770e449bd9e53b96f753176bf/users/c2ec23eb23ae4c1e8ddc37e73d759733/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:08.330130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77538-0fc8-42d1-bd78-415682f80a8a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:08.330420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77538-0fc8-42d1-bd78-415682f80a8a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:08.330733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e77538-0fc8-42d1-bd78-415682f80a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:08.351889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-64e77538-0fc8-42d1-bd78-415682f80a8a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:08.353222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-64e77538-0fc8-42d1-bd78-415682f80a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:08.413861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 779/1567] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:08.424013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:08.425732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:08.449457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:08.491338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83ee958-1705-476a-9637-19d307ce5a61 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:08.495101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:08.496213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:08.496607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:08.496750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:08.507084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:08.508576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:08.521147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 789/1568] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:10.095022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.095474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.095725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.109909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:10.125712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] MFA Rules not processed for user `c2ec23eb23ae4c1e8ddc37e73d759733`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:10.127006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:10.192657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 780/1569] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:10.197222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.207568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31155e6-dbb2-44fa-befd-ef5d0f85bba3 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.212178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.212795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.213054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.213327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.214482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.216509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.307115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 790/1570] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:10 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:10.308576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.319370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595441b2-423d-47c0-b3b2-a7fe214a9c05 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.380826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.381459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.381744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.382075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.382767 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.384701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.530650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 781/1571] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:10 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:10.535365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.550170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca7e2-03cf-4eda-a1a8-73e2d80c6358 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.555698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.556438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.556818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.557195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.558273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.560163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.581298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 791/1572] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:10.582733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.594778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf720d-3f7c-4832-be31-db0abef07d50 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.599704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.600484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.602649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.602649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] PATH_INFO: `/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b/users/6746662caa574234835e595130b9802b/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.620042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9296a1ae9d8d4bde88b599ac492dd37b, user_id=6746662caa574234835e595130b9802b, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.623398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.728091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 782/1573] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:10 2022] PUT /identity/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b/users/6746662caa574234835e595130b9802b/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:10.734486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.745043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8697be-b10f-4952-b04f-2fb6532fda87 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.809907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.810455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.810770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.811041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.811683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.813448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e987775f-1dba-4866-856b-6d1843073300 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.824556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 792/1574] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:10.828555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:10.838921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a0e87-cf64-48dc-a3d4-073e9a2c089d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:10.843794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:10.844323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:10.847521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:10.847521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] PATH_INFO: `/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b/users/6746662caa574234835e595130b9802b/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:10.847521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9296a1ae9d8d4bde88b599ac492dd37b, user_id=6746662caa574234835e595130b9802b, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:10.849533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:10.859855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 783/1575] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:10 2022] PUT /identity/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b/users/6746662caa574234835e595130b9802b/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:12.726839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:12.727309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:12.727309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:12.768483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] MFA Rules not processed for user `6746662caa574234835e595130b9802b`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:12.769760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:12.872998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 793/1576] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:12.882840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:12.884852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:12.887633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:12.889388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:12.904499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:12.940430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:12.966570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eebdaa-57e8-45eb-bf88-232b1e8fecc8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:12.967861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c79e0a8-de75-4037-a98a-b2d6da949cb8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:12.971691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:12.972328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:12.972607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:12.972879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] PATH_INFO: `/v3/users/033602599586468ca8cb69efdf3dd725` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:12.974114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] RBAC: Authorizing `identity:delete_user(user_id=033602599586468ca8cb69efdf3dd725)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:12.976149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:12.982088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:12.982895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:12.987425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:12.987425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:13.047543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:13.048983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043ea24c-0ec4-4e03-b1c4-8205a9d0e00b service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:13.060205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 794/1577] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:13.277082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '033602599586468ca8cb69efdf3dd725'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:13.293413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '033602599586468ca8cb69efdf3dd725'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:13.294133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '033602599586468ca8cb69efdf3dd725'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:13.301475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '033602599586468ca8cb69efdf3dd725'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:13.302181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '033602599586468ca8cb69efdf3dd725'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:13.434939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 784/1578] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:12 2022] DELETE /identity/v3/users/033602599586468ca8cb69efdf3dd725 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:13.435211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:13.435211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:13.435211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:13.454814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:13.456006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:13.515003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 795/1579] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:13.524821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:13.526302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:13.583720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bef73-5116-4b21-9b14-a053cafa6409 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:13.587480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:13.589773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:13.589773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:13.589773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:13.598608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:13.600145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:13.609867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 785/1580] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:14.080903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:14.082355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:14.095986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb56e5-35d2-4304-b300-df1e8a6790ca None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:14.102191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:14.103109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:14.105861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:14.105861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] PATH_INFO: `/v3/projects/e77ae4c4746b49ae8da4fcd08fb6d292` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:14.106359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77ae4c4746b49ae8da4fcd08fb6d292)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:14.108384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:14.248591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77ae4c4746b49ae8da4fcd08fb6d292', 'request_id': 'req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.341730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77ae4c4746b49ae8da4fcd08fb6d292', 'request_id': 'req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.343356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:14.348667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77ae4c4746b49ae8da4fcd08fb6d292', 'request_id': 'req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.361335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] The token cache is being invalidate because project e77ae4c4746b49ae8da4fcd08fb6d292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:14.361621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.362304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.364168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 796/1581] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:14 2022] DELETE /identity/v3/projects/e77ae4c4746b49ae8da4fcd08fb6d292 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:14.368288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:14.379780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c62dee-cf9f-4c30-ac53-5fa1c8c8cb2c service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:14.440270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:14.440898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:14.441179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:14.441482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] PATH_INFO: `/v3/users/5e3efadd3f714b3fab4f58c7a8f637b8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:14.442603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3efadd3f714b3fab4f58c7a8f637b8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:14.444614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:14.458172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:14.807729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3efadd3f714b3fab4f58c7a8f637b8'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.819834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3efadd3f714b3fab4f58c7a8f637b8'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.820513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3efadd3f714b3fab4f58c7a8f637b8'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.827805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3efadd3f714b3fab4f58c7a8f637b8'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.828438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3efadd3f714b3fab4f58c7a8f637b8'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:14.913532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 786/1582] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:14 2022] DELETE /identity/v3/users/5e3efadd3f714b3fab4f58c7a8f637b8 => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:15.585323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:15.587038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:15.603606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43fd82d-ac4c-40a5-b5be-9eb40dce72ac admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:15.671179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:15.671913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:15.672190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:15.672460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] PATH_INFO: `/v3/projects/59a1f04152c849d7a7963b0fd29b2f4d` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:15.673589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a1f04152c849d7a7963b0fd29b2f4d)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:15.675550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:15.755540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a1f04152c849d7a7963b0fd29b2f4d', 'request_id': 'req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:15.804578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a1f04152c849d7a7963b0fd29b2f4d', 'request_id': 'req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:15.811932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a1f04152c849d7a7963b0fd29b2f4d', 'request_id': 'req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:15.824601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] The token cache is being invalidate because project 59a1f04152c849d7a7963b0fd29b2f4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:15.824893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:15.825554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:15.827091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 797/1583] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:15 2022] DELETE /identity/v3/projects/59a1f04152c849d7a7963b0fd29b2f4d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:15.831378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:15.833005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:15.846092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336deb34-2d80-4fe6-85f5-b907b74a50c7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:15.908914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:15.909560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:15.909941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:15.910213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] PATH_INFO: `/v3/users/c7ee9ca87ed74eb583885a1d0904e63e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:15.911465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7ee9ca87ed74eb583885a1d0904e63e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:15.913529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:16.146570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7ee9ca87ed74eb583885a1d0904e63e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:16.159547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7ee9ca87ed74eb583885a1d0904e63e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:16.160239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7ee9ca87ed74eb583885a1d0904e63e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:16.169042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7ee9ca87ed74eb583885a1d0904e63e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:16.169766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-65f85fe5-f52d-4600-b166-97f882fc4d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7ee9ca87ed74eb583885a1d0904e63e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:16.307629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 787/1584] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:15 2022] DELETE /identity/v3/users/c7ee9ca87ed74eb583885a1d0904e63e => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:16.761009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:16.763518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:16.780401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489bc3c6-2345-4e49-b6fc-e1e64cb5ee17 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:16.905732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:16.907265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:16.907695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:16.908206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] PATH_INFO: `/v3/projects/eb176a2a41614c30b894e2787bfec4f7` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:16.909623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb176a2a41614c30b894e2787bfec4f7)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:16.913145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:16.942272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:17.048022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb176a2a41614c30b894e2787bfec4f7', 'request_id': 'req-deb92e52-1f1a-4dbe-a954-539a600fd32d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:17.119030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb176a2a41614c30b894e2787bfec4f7', 'request_id': 'req-deb92e52-1f1a-4dbe-a954-539a600fd32d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:17.128232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb176a2a41614c30b894e2787bfec4f7', 'request_id': 'req-deb92e52-1f1a-4dbe-a954-539a600fd32d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:17.147064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] The token cache is being invalidate because project eb176a2a41614c30b894e2787bfec4f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:17.147925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:17.149164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:17.154955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 798/1585] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:16 2022] DELETE /identity/v3/projects/eb176a2a41614c30b894e2787bfec4f7 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:17.156389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72d974-7853-4346-a8a2-70feb357404b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.156833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72d974-7853-4346-a8a2-70feb357404b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.156956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72d974-7853-4346-a8a2-70feb357404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.195576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.197957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:17.201659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4a72d974-7853-4346-a8a2-70feb357404b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:17.202932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4a72d974-7853-4346-a8a2-70feb357404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:17.207533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:17.280990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb92e52-1f1a-4dbe-a954-539a600fd32d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.287516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 788/1586] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:17.292915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.292915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.292915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.295636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.296449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.297289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.297567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.299719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:17.323115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:17.325108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:17.337384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 789/1587] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:17.343215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.356563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e97b69c-da56-460e-ae81-da2d3542582f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.361926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.362553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.363735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.364010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.364658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:17.364960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.366554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.383660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 790/1588] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:05:17.384398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.395337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59988961-0850-4ee5-9036-8010ee017c8b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.395988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.397643 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.400509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.401242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.401522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.402425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.403171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.405236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.409397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 799/1589] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:17 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:17.512500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 791/1590] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:17 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:17.513729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.523999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c72047-b906-4abc-9957-08a097908cc6 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.615388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:17.632060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.634292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.634596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.636393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.637067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.642013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.760539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 800/1591] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:17 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:17.768024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.779594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc261-07f9-4627-84cb-6900cc3523d5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.784239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.784776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.785051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.785317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.785949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.787737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.799300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 792/1592] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:17.803406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.815065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537464e6-640a-487a-a4ae-4521470cce61 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.820512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.821291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.821694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.822061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.822835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.825787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ff8d3f-eff9-491e-981c-2d44b50e1717 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.840122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 801/1593] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:17.843978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:17.857070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14746246-3ddd-4ca1-9d11-5fa5ecaa8ac5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:17.861840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:17.862514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.862814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.863096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] PATH_INFO: `/v3/projects/4f03c1cd40ef4f3abc89c719723d2646/users/4292635778d1494eae3dec09989e156e/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:17.884996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f03c1cd40ef4f3abc89c719723d2646, user_id=4292635778d1494eae3dec09989e156e, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:17.888897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:17.979510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 793/1594] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:17 2022] PUT /identity/v3/projects/4f03c1cd40ef4f3abc89c719723d2646/users/4292635778d1494eae3dec09989e156e/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:17.997647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:17.997647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:17.997647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:18.064727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] MFA Rules not processed for user `4292635778d1494eae3dec09989e156e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:18.072470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:18.197195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 802/1595] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:18.202497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:18.336393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf55fd1d-297a-4710-8bc9-d8d7f57db856 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:18.336393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:18.336393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:18.336393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:18.336959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:18.336959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:18.336959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:18.695717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:18.701829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 794/1596] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:18 2022] POST /identity/v3/projects => generated 383 bytes in 496 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:18.707983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20a7290-0c3b-41c2-8f85-0c42b64c670d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:18.780320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:18.780959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:18.781310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:18.781588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:18.782232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:18.784415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:19.143613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 803/1597] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:18 2022] POST /identity/v3/users => generated 292 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:19.148518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:19.160641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45f2d5-5ad5-4490-adc3-c5320278a424 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:19.165490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:19.169420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.169420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.169420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.169420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:19.170147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:19.183308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 795/1598] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:19.187780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:19.198337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783f3e47-f5b3-43ee-bc14-74f63f17295c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:19.203948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:19.204797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.205269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.205638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.206452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:19.208072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08fc21e-f2a9-4a32-8150-88a463863bac admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:19.219835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 804/1599] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:19.235972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:19.254909 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c5dac-8365-439a-9580-a13a21e90000 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:19.260513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:19.261311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.261940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.262221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] PATH_INFO: `/v3/projects/ee50ceb6e38b483ebc4443911bfd6531/users/2a03a05d3b6843a3894fc6e03ed94503/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.264735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:19.279614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee50ceb6e38b483ebc4443911bfd6531, user_id=2a03a05d3b6843a3894fc6e03ed94503, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:19.282158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:19.406899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 796/1600] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:19 2022] PUT /identity/v3/projects/ee50ceb6e38b483ebc4443911bfd6531/users/2a03a05d3b6843a3894fc6e03ed94503/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:19.417463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7762-1c10-4195-813d-b329392add2c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.417775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7762-1c10-4195-813d-b329392add2c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.418065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095c7762-1c10-4195-813d-b329392add2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.447536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-095c7762-1c10-4195-813d-b329392add2c None None] MFA Rules not processed for user `2a03a05d3b6843a3894fc6e03ed94503`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:19.449497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-095c7762-1c10-4195-813d-b329392add2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:19.493450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:19.539276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 805/1601] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:19.540834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:19.609945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27d79d-dfdd-4686-bb5e-4ec40989b282 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:19.614452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:19.615265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.615562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.615813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.616443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:19.618339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:19.744549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 797/1602] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:19 2022] POST /identity/v3/projects => generated 385 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:19.749265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7762-1c10-4195-813d-b329392add2c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:19.760945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095c7762-1c10-4195-813d-b329392add2c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:19.839069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:19.842165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:19.842165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:19.842165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:19.842165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:19.848898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.013290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 806/1603] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:19 2022] POST /identity/v3/users => generated 299 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:20.017910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:20.032268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668aefa6-4250-4ce2-aa45-32a29d6c0d28 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:20.039877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:20.041618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.043356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.043356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.044104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:20.046528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.062255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 798/1604] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:20.066416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:20.085699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eba199-4d7d-4d60-9410-c42041dfd056 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:20.085699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:20.085699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.085699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.086070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] PATH_INFO: `/v3/projects/b16ed6d1926940b6ade836a2a6a60124/users/f52a7963b21a4c8b99e24d21b5c799ed/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.106306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16ed6d1926940b6ade836a2a6a60124, user_id=f52a7963b21a4c8b99e24d21b5c799ed, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:20.109473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.262569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 807/1605] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:20 2022] PUT /identity/v3/projects/b16ed6d1926940b6ade836a2a6a60124/users/f52a7963b21a4c8b99e24d21b5c799ed/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:20.267825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:20.286163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa030bb7-0d68-4720-a85f-5a2d4e2a238b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:20.374346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:20.375050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.375354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.375634 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.376574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:20.378791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23531f9-61da-4410-ab3f-a71ab1ab9efe admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.391305 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 799/1606] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:20.395409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:20.408059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b8360-e1de-446d-bb73-19d5b438410c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:20.413543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:20.414285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.414948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.415293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] PATH_INFO: `/v3/projects/b16ed6d1926940b6ade836a2a6a60124/users/f52a7963b21a4c8b99e24d21b5c799ed/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.417548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16ed6d1926940b6ade836a2a6a60124, user_id=f52a7963b21a4c8b99e24d21b5c799ed, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:20.421051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.432305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 808/1607] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:20 2022] PUT /identity/v3/projects/b16ed6d1926940b6ade836a2a6a60124/users/f52a7963b21a4c8b99e24d21b5c799ed/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:20.443908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.444284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.444601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.483494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] MFA Rules not processed for user `f52a7963b21a4c8b99e24d21b5c799ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:20.488003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:20.534156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:20.563779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 800/1608] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:20.576116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:20.635534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e0ec1-37d5-4a1e-b092-b8fe9f241d69 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:20.639851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:20.640583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:20.640969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:20.641367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:20.688122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:20.711092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:20.712549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67db7912-4ad6-43d5-bd17-5a015a5dc86b service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:20.722439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 809/1609] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:21.943590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:21.945035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:22.034534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc400b3-7061-4dea-93c8-e9a4c0c94a33 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:22.039369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:22.040432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:22.040714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:22.040985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] PATH_INFO: `/v3/users/a488f48c5698496ab92983604ccfc257` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:22.042205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a488f48c5698496ab92983604ccfc257)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:22.044828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:22.313330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a488f48c5698496ab92983604ccfc257'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:22.325838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a488f48c5698496ab92983604ccfc257'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:22.326584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a488f48c5698496ab92983604ccfc257'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:22.334620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a488f48c5698496ab92983604ccfc257'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:22.335372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a488f48c5698496ab92983604ccfc257'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:22.428643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 801/1610] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:21 2022] DELETE /identity/v3/users/a488f48c5698496ab92983604ccfc257 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:22.432964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:22.433477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:22.433812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:22.456388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:22.457987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:22.520085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 810/1611] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:22.530717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:22.532333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:22.592545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb274a-5032-4771-bd74-4bb31c02d902 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:22.596488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:22.597097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:22.597375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:22.597645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:22.607067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:22.609205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:22.614601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:22.620279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 802/1612] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:23.132140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.133554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.146172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ce20c-6afa-414b-98ef-ec6cec917513 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.151007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.151730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.152074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.152419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] PATH_INFO: `/v3/projects/2fa115142e01451f9a327780d4f0ca8f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.153797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa115142e01451f9a327780d4f0ca8f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.156542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:23.271872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.272692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.300620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa115142e01451f9a327780d4f0ca8f', 'request_id': 'req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:23.363811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f0310-9b6a-4dfa-a8e5-8c18cd647b7c service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.421768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa115142e01451f9a327780d4f0ca8f', 'request_id': 'req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:23.423954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.426352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.426352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.426352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.427119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.429205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa115142e01451f9a327780d4f0ca8f', 'request_id': 'req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:23.445448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] The token cache is being invalidate because project 2fa115142e01451f9a327780d4f0ca8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:23.446004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:23.446550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a52fd77f-0ee3-4dde-b3c2-b9d570ed3804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:23.449469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 811/1613] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:23 2022] DELETE /identity/v3/projects/2fa115142e01451f9a327780d4f0ca8f => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:23.454493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.454831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.455107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.478508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:23.480018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.499609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.501049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9ae923-2089-4b42-b730-e8290bacd785 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:23.510198 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 803/1614] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:23.551676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 812/1615] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:23.566727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.567017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.567293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.587978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:23.589195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.604424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 804/1616] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:23.605594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.612116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:23.618335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33f52e-825a-43b1-8731-196a1db6d428 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.624820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.627170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.627170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.627170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.627170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:23.627170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.628540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:23.640970 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 813/1617] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:05:23.645476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.655711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60a2c2-cd9e-460d-a39c-5817cb0335c2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.660317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.660913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.661189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.661461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.662092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.664011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:23.797644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.799339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:23.812244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 805/1618] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:23 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:23.813613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e5d260-13d7-4bf3-bedc-982b50b7de57 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.825864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:23.879644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.880290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.880572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.880841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] PATH_INFO: `/v3/users/88da08adefdf40d583e73b7ff395dd8f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.881530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:23.882202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88da08adefdf40d583e73b7ff395dd8f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.884853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:23.906050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d7b85e-9771-496f-b8da-1ce158956478 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:23.911810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:23.912641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:23.913033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:23.913356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:23.913995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:23.916253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.008187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 806/1619] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:23 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:24.012358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:24.022904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebb071e-b622-4677-97dd-bb97302763fe admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:24.027730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:24.029133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.029413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.029689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.030328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:24.032045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.042481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 807/1620] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:24.046694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:24.056297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bf069d-26b4-4323-b4bf-6958ab07383b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:24.061069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:24.061646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.061923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.062194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.062853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:24.064431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.076843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 808/1621] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:24.080862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:24.090610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040696e-1f99-4ec9-887b-9097e228a101 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:24.096030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:24.096648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.096928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.097202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] PATH_INFO: `/v3/projects/680f20eb6fd54decb74bf6275c7cc12d/users/fca30433e2eb4baa8b0bff97cb899205/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.111706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=680f20eb6fd54decb74bf6275c7cc12d, user_id=fca30433e2eb4baa8b0bff97cb899205, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:24.114322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97da3d3-80ed-46d2-87e6-bfc2772ff39e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.138692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88da08adefdf40d583e73b7ff395dd8f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:24.139423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88da08adefdf40d583e73b7ff395dd8f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:24.151359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88da08adefdf40d583e73b7ff395dd8f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:24.205675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 809/1622] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:24 2022] PUT /identity/v3/projects/680f20eb6fd54decb74bf6275c7cc12d/users/fca30433e2eb4baa8b0bff97cb899205/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:24.215899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.216231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.216568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.243436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] MFA Rules not processed for user `fca30433e2eb4baa8b0bff97cb899205`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:24.244645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:24.261874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88da08adefdf40d583e73b7ff395dd8f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:24.262569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-e7977d09-8870-47a4-8f3d-ec15745fbec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88da08adefdf40d583e73b7ff395dd8f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:24.278244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 814/1623] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:23 2022] DELETE /identity/v3/users/88da08adefdf40d583e73b7ff395dd8f => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:24.282985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.283270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.283542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.303388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:24.304595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:24.314792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 810/1624] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:24.348841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:24.350299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:24.387646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 815/1625] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:24.399130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:24.400543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:24.412707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b3687f-cfbe-4a41-a23f-c3fde57f5084 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:24.458954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d11bc21-2e3a-469c-a784-7097ee62d3aa None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:24.462938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:24.463563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.463727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:24.463892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.464176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.464631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:24.465021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:24.465524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:24.476805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:24.478260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.489758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 816/1626] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:24.541257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:24.542808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:24.554058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 811/1627] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:24.736684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:25.085358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:25.093556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:25.095142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:25.109118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ff126-2725-4da3-b21c-4ab828e44d18 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:25.114236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:25.114899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:25.115180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:25.115451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] PATH_INFO: `/v3/projects/b0e8a6692b744a4aa6353f0247c4fcce` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:25.116708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0e8a6692b744a4aa6353f0247c4fcce)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:25.118550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:25.240268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0e8a6692b744a4aa6353f0247c4fcce', 'request_id': 'req-5a7917ab-162e-4efb-b906-d0c0618ee161'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.364963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0e8a6692b744a4aa6353f0247c4fcce', 'request_id': 'req-5a7917ab-162e-4efb-b906-d0c0618ee161'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.372537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0e8a6692b744a4aa6353f0247c4fcce', 'request_id': 'req-5a7917ab-162e-4efb-b906-d0c0618ee161'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.395530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] The token cache is being invalidate because project b0e8a6692b744a4aa6353f0247c4fcce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:25.396325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.397001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.398721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 817/1628] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:25 2022] DELETE /identity/v3/projects/b0e8a6692b744a4aa6353f0247c4fcce => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:25.402624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:25.415246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd2e87-c888-45e3-b0ca-d0fba8db39c7 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:25.475715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:25.476459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:25.476811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:25.477170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] PATH_INFO: `/v3/users/1d9ce75471c344e3bc38dec2ea2d9f7a` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:25.478390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d9ce75471c344e3bc38dec2ea2d9f7a)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:25.480419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:25.584775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:25.586236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:25.657222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7917ab-162e-4efb-b906-d0c0618ee161 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:25.661104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:25.661724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:25.662010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:25.662285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:25.664058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:25.733195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:25.734732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:25.745953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 818/1629] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:25.790977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d9ce75471c344e3bc38dec2ea2d9f7a'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.804215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d9ce75471c344e3bc38dec2ea2d9f7a'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.804990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d9ce75471c344e3bc38dec2ea2d9f7a'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.812026 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d9ce75471c344e3bc38dec2ea2d9f7a'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.812680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b0cd0705-a7bf-4199-ae2d-4bbc36dd94f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d9ce75471c344e3bc38dec2ea2d9f7a'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:25.983326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 812/1630] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:25 2022] DELETE /identity/v3/users/1d9ce75471c344e3bc38dec2ea2d9f7a => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:26.493181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:26.496823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:26.510186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131f0eaa-7948-4180-bfa2-5e01445d6446 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:26.584747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:26.585417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:26.586000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:26.586000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] PATH_INFO: `/v3/projects/8162be9f54e645049234d1d309c8ffd8` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:26.587205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8162be9f54e645049234d1d309c8ffd8)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:26.589627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:26.795828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8162be9f54e645049234d1d309c8ffd8', 'request_id': 'req-d16faf3a-01bc-4c3d-bd1e-a34057f90707'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:26.907541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8162be9f54e645049234d1d309c8ffd8', 'request_id': 'req-d16faf3a-01bc-4c3d-bd1e-a34057f90707'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:26.913569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8162be9f54e645049234d1d309c8ffd8', 'request_id': 'req-d16faf3a-01bc-4c3d-bd1e-a34057f90707'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:26.925621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] The token cache is being invalidate because project 8162be9f54e645049234d1d309c8ffd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:26.925922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:26.926592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d16faf3a-01bc-4c3d-bd1e-a34057f90707 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:26.931750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 819/1631] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:26 2022] DELETE /identity/v3/projects/8162be9f54e645049234d1d309c8ffd8 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:26.933454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:26.933969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:26.934488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:26.946707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:26.963373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:26.964648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:27.030418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 813/1632] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:27.039271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.040154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.040426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.070453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:27.070453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:27.075512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 820/1633] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:27.080870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.091080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7956d-602f-44f9-a7f4-97592f0f0f22 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.096661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.097281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.097558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.097827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.098476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:27.098828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.100413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.123599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 814/1634] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:27.125080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.142915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c539c4-4e78-4916-ae9b-86d37807e92e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.159442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.159442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.159442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.159442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.159740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.159740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.193337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:27.271557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 821/1635] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:27 2022] POST /identity/v3/projects => generated 365 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:27.275357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.285420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc9966-75d9-42de-b3e8-69f31ce3d96d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.343146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.343763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.344140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.344418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.345053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.347001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.486374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 815/1636] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:27 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:27.490751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.502235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f471c-f473-4015-8693-bda472508d6a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.506975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.507611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.507908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.508484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.508896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.510485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.522740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 822/1637] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:27.533413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.542797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e7ba12-0168-4a27-a8a6-667e4dcb9235 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.548443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.549055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.549457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.549732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.550392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.552235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14482f78-efbb-4a3d-a0be-4ea079e2cfc8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.565202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 816/1638] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:27.569575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.579395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c631826-d76b-41e6-99e6-d56f555bc215 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.583863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.584500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.584775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.585043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] PATH_INFO: `/v3/projects/2e47bdd8e76e480d894bf4691a9bc1c0/users/25281988dfa14e4683c9366d2d4dd7e1/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.599529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e47bdd8e76e480d894bf4691a9bc1c0, user_id=25281988dfa14e4683c9366d2d4dd7e1, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.602291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:27.730173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 823/1639] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:27 2022] PUT /identity/v3/projects/2e47bdd8e76e480d894bf4691a9bc1c0/users/25281988dfa14e4683c9366d2d4dd7e1/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:27.742185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.742490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.742817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.770076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] MFA Rules not processed for user `25281988dfa14e4683c9366d2d4dd7e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:27.771309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:27.835875 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 817/1640] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:27.840485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:27.904798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7804e6a-56c3-4099-8474-431b0f20b806 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:27.909474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:27.910082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:27.910365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:27.910654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:27.911308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:27.913216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.025252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 824/1641] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:27 2022] POST /identity/v3/projects => generated 365 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:28.029894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.040569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1236a-4975-4698-9af3-e0ed3c256242 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.108195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.109071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.109556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.109979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.110820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.113669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.159697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.161420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:28.175345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9548951e-6c75-4d45-8283-5583dde4c864 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.180395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.181033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.181511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.181787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] PATH_INFO: `/v3/users/fca30433e2eb4baa8b0bff97cb899205` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.182974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca30433e2eb4baa8b0bff97cb899205)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.184923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.216845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 818/1642] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:28 2022] POST /identity/v3/users => generated 289 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:28.221050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.231012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a9802-e10a-46d1-aabe-5a9c05921422 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.236340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.237009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.237465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.237581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.238202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.239926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.250034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 819/1643] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:28.254187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.264278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882ed2-6526-4060-9592-275531ad02dd admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.268960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.269592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.269873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.270145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] PATH_INFO: `/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f/users/936625288b1844ea92199f22b553d8cf/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.285978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14e2329145d4f0f9ffa2982d3a42f3f, user_id=936625288b1844ea92199f22b553d8cf, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.288706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.379185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca30433e2eb4baa8b0bff97cb899205'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:28.379915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca30433e2eb4baa8b0bff97cb899205'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:28.393470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca30433e2eb4baa8b0bff97cb899205'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:28.405876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 820/1644] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:28 2022] PUT /identity/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f/users/936625288b1844ea92199f22b553d8cf/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:28.410039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.472959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e830d38d-8377-427e-94a4-68d0ef74a44a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.477556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.478143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.478419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.478719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.479363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.480958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.491381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 821/1645] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:28.501219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.510731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510978a-a8b8-468d-815a-880ce150f0dc admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.512817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca30433e2eb4baa8b0bff97cb899205'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:28.514235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ea22c19f-269a-4261-b389-fc0867461dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca30433e2eb4baa8b0bff97cb899205'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:28.516414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.517088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.517374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.517705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] PATH_INFO: `/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f/users/936625288b1844ea92199f22b553d8cf/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.519694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14e2329145d4f0f9ffa2982d3a42f3f, user_id=936625288b1844ea92199f22b553d8cf, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.522399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a95861-d48d-4692-a43e-2d7988905690 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.531008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 825/1646] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:28 2022] DELETE /identity/v3/users/fca30433e2eb4baa8b0bff97cb899205 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:28.532389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.532682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.532961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.534724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 822/1647] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:28 2022] PUT /identity/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f/users/936625288b1844ea92199f22b553d8cf/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:28.545811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.546090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.546408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.553753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:28.555779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:28.576045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] MFA Rules not processed for user `936625288b1844ea92199f22b553d8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:28.577354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:28.615797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 826/1648] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:28.626899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.630377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:28.647164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 823/1649] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:28.659140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:28.660570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:28.681791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:28.690561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5fafa-8da4-421c-9af3-26b9a368987d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.694273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.694933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.695215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.695490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.705169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.706696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.716941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 827/1650] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:28.719624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ac8b54-987d-49ea-a600-6922fdeb3f96 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:28.723377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:28.724139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:28.724507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:28.724773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:28.735411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:28.736892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4296da-ff0f-44ac-9cbb-a6e10d948921 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:28.746680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 824/1651] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:28 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:29.027852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:29.286223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:29.299199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:29.313780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15901dee-3f7d-4532-9b33-6cd053c6485b service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:29.319153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:29.320061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:29.320542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:29.321046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] PATH_INFO: `/v3/projects/680f20eb6fd54decb74bf6275c7cc12d` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:29.322492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=680f20eb6fd54decb74bf6275c7cc12d)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:29.325050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:29.462208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:29.505598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '680f20eb6fd54decb74bf6275c7cc12d', 'request_id': 'req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:29.616183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '680f20eb6fd54decb74bf6275c7cc12d', 'request_id': 'req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:29.624188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '680f20eb6fd54decb74bf6275c7cc12d', 'request_id': 'req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:29.638030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] The token cache is being invalidate because project 680f20eb6fd54decb74bf6275c7cc12d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:29.638375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:29.639071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6c6e228f-7bc2-4e42-b7eb-d7fe43b6a5fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:29.643009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 828/1652] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:29 2022] DELETE /identity/v3/projects/680f20eb6fd54decb74bf6275c7cc12d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:29.648192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:29.648484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:29.648760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:29.670138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:29.671613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:29.734841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 825/1653] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:29.746071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:29.746500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:29.750125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:29.770386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:29.773286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:29.780909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 829/1654] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:29.785112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:29.795048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920e9e9-c1ef-4014-a361-939fc3bebe69 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:29.803674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:29.804294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:29.804568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:29.807827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:29.807827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:29.807827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:29.807827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:29.824702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 826/1655] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:05:29.826772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:29.836522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46279351-27f6-45f1-bdde-f3f78576191f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:29.849264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:29.849896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:29.850175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:29.850444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:29.851117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:29.853031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:29.996332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 830/1656] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:29 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:30.001292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.011944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6522a6-dabf-4cbd-820b-742b466baf86 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.089473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.090093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.090372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.090671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.091950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.093862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.210154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.211862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.213042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 827/1657] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:30 2022] POST /identity/v3/users => generated 284 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:30.219972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.225962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5680857a-3aad-439d-8b81-1649513ba6a1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.229685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b4ffa-78d8-4021-b55f-23d7054f596c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.230812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.231506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.231813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.232588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] PATH_INFO: `/v3/users/25281988dfa14e4683c9366d2d4dd7e1` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.233679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25281988dfa14e4683c9366d2d4dd7e1)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.234570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.235196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.235477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.235750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.235848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.236444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.238095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.250615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 828/1658] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:30.254664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.264970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd58a35-0133-45b1-a7bf-2417e08a7535 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.269771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.270362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.270653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.270940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.271575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.273229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.283794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 829/1659] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:30.287858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.298195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afa2e9-7f7a-4968-9886-f3994a689e1c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.302720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.303378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.303655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.303928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] PATH_INFO: `/v3/projects/c2352ca767e244f9a63cd48d65e491b6/users/bfd7118cba6a46f79f520bfe197b53dc/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.320663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2352ca767e244f9a63cd48d65e491b6, user_id=bfd7118cba6a46f79f520bfe197b53dc, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.323637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca05ee76-9f26-46a4-926e-bf49f2b0757f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.418960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25281988dfa14e4683c9366d2d4dd7e1'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:30.420257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25281988dfa14e4683c9366d2d4dd7e1'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:30.432631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25281988dfa14e4683c9366d2d4dd7e1'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:30.461939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 830/1660] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:30 2022] PUT /identity/v3/projects/c2352ca767e244f9a63cd48d65e491b6/users/bfd7118cba6a46f79f520bfe197b53dc/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:30.475620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18f55c-2dcb-4d38-8c42-58c413e25b66 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.476179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18f55c-2dcb-4d38-8c42-58c413e25b66 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.476179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c18f55c-2dcb-4d38-8c42-58c413e25b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.506297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9c18f55c-2dcb-4d38-8c42-58c413e25b66 None None] MFA Rules not processed for user `bfd7118cba6a46f79f520bfe197b53dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:30.507566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9c18f55c-2dcb-4d38-8c42-58c413e25b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.564585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25281988dfa14e4683c9366d2d4dd7e1'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:30.565246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25281988dfa14e4683c9366d2d4dd7e1'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:30.580498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 831/1661] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:30.582219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 831/1662] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:30 2022] DELETE /identity/v3/users/25281988dfa14e4683c9366d2d4dd7e1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:30.586742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.587040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.587315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.594682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.596267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.607393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:30.609803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.681825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 832/1663] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:30.692936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f408a703-0554-4d66-ab36-de14c2cc69b5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.695020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:30.696898 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.697410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.698260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.698664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.698951 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.702119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:30.710272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:30.762461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea0fe9-8ae2-4958-9be9-ea34a0998dee None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:30.766678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:30.767381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:30.767665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:30.767939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:30.774247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.776153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.778007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:30.779940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:30.781420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:30.788412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 832/1664] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:30.792477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 833/1665] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:31.335407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:31.337082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:31.367932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c014d67-4a31-4f04-8146-67353103a04b service glance] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:31.367932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:31.367932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:31.367932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:31.368487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] PATH_INFO: `/v3/projects/2e47bdd8e76e480d894bf4691a9bc1c0` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:31.368487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e47bdd8e76e480d894bf4691a9bc1c0)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:31.368487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:31.525530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e47bdd8e76e480d894bf4691a9bc1c0', 'request_id': 'req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:31.718854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e47bdd8e76e480d894bf4691a9bc1c0', 'request_id': 'req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:31.725130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e47bdd8e76e480d894bf4691a9bc1c0', 'request_id': 'req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:31.737817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] The token cache is being invalidate because project 2e47bdd8e76e480d894bf4691a9bc1c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:31.738112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:31.738801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:31.740333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 833/1666] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:31 2022] DELETE /identity/v3/projects/2e47bdd8e76e480d894bf4691a9bc1c0 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:31.744486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:31.754499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ffb287-1a60-4460-8b8b-96a981a621ef service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:31.819340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:31.820216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:31.820702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:31.821116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] PATH_INFO: `/v3/users/936625288b1844ea92199f22b553d8cf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:31.822951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] RBAC: Authorizing `identity:delete_user(user_id=936625288b1844ea92199f22b553d8cf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:31.825928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:31.970193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:31.999666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:32.003188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:32.018404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2236b3-3f4a-445f-80dc-8ab421d83bf3 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:32.024190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:32.025169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:32.025629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:32.026373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] PATH_INFO: `/v3/users/4292635778d1494eae3dec09989e156e` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:32.028321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4292635778d1494eae3dec09989e156e)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:32.031597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:32.065105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '936625288b1844ea92199f22b553d8cf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.082742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '936625288b1844ea92199f22b553d8cf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.083541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '936625288b1844ea92199f22b553d8cf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.093847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '936625288b1844ea92199f22b553d8cf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.094882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-b9ebeed6-bca9-4f5f-b4a4-a16f4a84f00b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '936625288b1844ea92199f22b553d8cf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.172877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 834/1667] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:31 2022] DELETE /identity/v3/users/936625288b1844ea92199f22b553d8cf => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:32.253472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4292635778d1494eae3dec09989e156e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.254218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4292635778d1494eae3dec09989e156e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.268538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4292635778d1494eae3dec09989e156e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.444934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4292635778d1494eae3dec09989e156e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.445536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4292635778d1494eae3dec09989e156e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:32.460161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 834/1668] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:31 2022] DELETE /identity/v3/users/4292635778d1494eae3dec09989e156e => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:32.464900 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:32.465192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:32.465472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:32.484930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:32.486164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:32.556362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 835/1669] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:32.570115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:32.571673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:32.638975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a251a33-8951-4046-885c-ef7196647094 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:32.644229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:32.644855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:32.645135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:32.645409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:32.656287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:32.657191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c21c87-4e13-48fa-ba70-068d21525eae service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:32.668627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 835/1670] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:32.745547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:32.749630 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:32.766998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb88913-484b-4776-8c41-e6ba16fae158 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:32.788219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:32.788219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:32.788219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:32.788219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] PATH_INFO: `/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:32.788717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14e2329145d4f0f9ffa2982d3a42f3f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:32.788717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:32.944476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14e2329145d4f0f9ffa2982d3a42f3f', 'request_id': 'req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.056982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14e2329145d4f0f9ffa2982d3a42f3f', 'request_id': 'req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.066215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14e2329145d4f0f9ffa2982d3a42f3f', 'request_id': 'req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.089153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] The token cache is being invalidate because project f14e2329145d4f0f9ffa2982d3a42f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:33.089448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.090749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36c3302-1312-4ffb-a9ee-2d9e8f3b8efd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.093422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 836/1671] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:32 2022] DELETE /identity/v3/projects/f14e2329145d4f0f9ffa2982d3a42f3f => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:33.098403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.099281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.099569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.127008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:33.128414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:33.235032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 836/1672] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:33.239362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.239650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.249619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.272679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:33.274119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:33.302001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 837/1673] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:33.306087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.318953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22712e5-4366-48af-96b4-e851b62ec91b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.329456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.330281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.330819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.331151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.331811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:33.332140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.333882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.350084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 837/1674] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:05:33.352940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:33.354582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.365315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803b3045-6a30-45de-99c7-098c35a935cb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.370484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.371380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.371668 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.371964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.373140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.374962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.525286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:33.536913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.538714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:33.542486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 838/1675] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:33 2022] POST /identity/v3/projects => generated 365 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:33.543770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.555901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8db3fb-04b6-4b3f-b33f-8ffd74241f5c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.626471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb74b40-09b1-48e5-8c9f-8cbcc6816edb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.632622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.633419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.633823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.634295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.635099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.637327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.647767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.648543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.648897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.649168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] PATH_INFO: `/v3/projects/4f03c1cd40ef4f3abc89c719723d2646` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.650319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f03c1cd40ef4f3abc89c719723d2646)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.652277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.847588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 839/1676] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:33 2022] POST /identity/v3/users => generated 283 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:33.854388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.869284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e3049e-2955-4ce3-9553-fde284ad1f06 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.881978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f03c1cd40ef4f3abc89c719723d2646', 'request_id': 'req-76cddc77-b8fa-4a59-8d95-5b726819e024'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.945368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.946067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.946343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.946613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.947294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.955128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.960977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 840/1677] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:33.964973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:33.975583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054826bc-50b5-4e59-8ec6-314d020963d7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:33.980661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:33.981266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:33.981453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f03c1cd40ef4f3abc89c719723d2646', 'request_id': 'req-76cddc77-b8fa-4a59-8d95-5b726819e024'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.981561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:33.981965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:33.982656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:33.984288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:33.989164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f03c1cd40ef4f3abc89c719723d2646', 'request_id': 'req-76cddc77-b8fa-4a59-8d95-5b726819e024'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:33.995286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 841/1678] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:34.000130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.003298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] The token cache is being invalidate because project 4f03c1cd40ef4f3abc89c719723d2646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:34.003589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.004349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.006561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 838/1679] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:33 2022] DELETE /identity/v3/projects/4f03c1cd40ef4f3abc89c719723d2646 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:34.009685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.009838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abf17-2a28-4790-bcb2-93014271cd21 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.011235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:34.016536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.017189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.017470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.017742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] PATH_INFO: `/v3/projects/665f96c7ad5344b2a7f735c5fcc8d650/users/628f9f646c934277963412611b083515/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.024867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cddc77-b8fa-4a59-8d95-5b726819e024 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.029874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.030512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.030821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.031099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] PATH_INFO: `/v3/users/2a03a05d3b6843a3894fc6e03ed94503` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.032284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a03a05d3b6843a3894fc6e03ed94503)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.033073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665f96c7ad5344b2a7f735c5fcc8d650, user_id=628f9f646c934277963412611b083515, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.034408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.035830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c8ced-0e89-48e8-8b49-6622f53f2657 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.113158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 842/1680] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:33 2022] PUT /identity/v3/projects/665f96c7ad5344b2a7f735c5fcc8d650/users/628f9f646c934277963412611b083515/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:34.130260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.130701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.130983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.162459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] MFA Rules not processed for user `628f9f646c934277963412611b083515`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:34.163738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:34.255842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 843/1681] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:34.260221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.261499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:34.298950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a03a05d3b6843a3894fc6e03ed94503'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.299621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a03a05d3b6843a3894fc6e03ed94503'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.315469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a03a05d3b6843a3894fc6e03ed94503'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.335606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c770c14-51da-4267-8f71-aa6c418b9d26 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.340190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.342800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.342800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.342800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.342800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.344136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.387416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a03a05d3b6843a3894fc6e03ed94503'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.388004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a03a05d3b6843a3894fc6e03ed94503'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:34.401065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 839/1682] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:34 2022] DELETE /identity/v3/users/2a03a05d3b6843a3894fc6e03ed94503 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:34.459801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 844/1683] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:34 2022] POST /identity/v3/projects => generated 363 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:34.476361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.477934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:34.543424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956389b-48b8-4cd4-846e-d23d765417de admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.601689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.602299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.602575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.602878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.603510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.605455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.755176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 840/1684] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:34 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:34.759248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.769463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f8dac-a6f2-464e-9f72-d8875fa4e60b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.774037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.774695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.775032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.775307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.775953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.777554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.788868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 845/1685] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:34.795471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.804410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ff23b-9309-4a4c-b716-18130223a66b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.808952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.809605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.809961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.810318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] PATH_INFO: `/v3/projects/474ba0e28d0b467db9e7221dde482308/users/c81819ed068d450583a1d66107aa85ab/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.824834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474ba0e28d0b467db9e7221dde482308, user_id=c81819ed068d450583a1d66107aa85ab, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.827665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.886880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:34.888966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:34.914471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37653793-7a99-4fd8-879d-43e942307971 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:34.921301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:34.921947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:34.922223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:34.929802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] PATH_INFO: `/v3/projects/ee50ceb6e38b483ebc4443911bfd6531` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:34.929802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee50ceb6e38b483ebc4443911bfd6531)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:34.929802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:34.971290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 841/1686] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:34 2022] PUT /identity/v3/projects/474ba0e28d0b467db9e7221dde482308/users/c81819ed068d450583a1d66107aa85ab/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:34.975386 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:35.042414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f200eb0-e548-4c9a-8eca-41869de21519 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:35.050616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:35.051331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.051639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.051909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.052553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:35.054146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:35.065642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 842/1687] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:34 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:35.069460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:35.081054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b16f4-5de6-4706-82e7-5d14970a1690 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:35.094885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee50ceb6e38b483ebc4443911bfd6531', 'request_id': 'req-ec2adc95-34f2-4401-a6c8-8fb43d006e42'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.154359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:35.155062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.155346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.155650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] PATH_INFO: `/v3/projects/474ba0e28d0b467db9e7221dde482308/users/c81819ed068d450583a1d66107aa85ab/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.157588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474ba0e28d0b467db9e7221dde482308, user_id=c81819ed068d450583a1d66107aa85ab, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:35.160303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb92f-884b-4458-8f99-1d6bf9478717 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:35.165588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee50ceb6e38b483ebc4443911bfd6531', 'request_id': 'req-ec2adc95-34f2-4401-a6c8-8fb43d006e42'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.172902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee50ceb6e38b483ebc4443911bfd6531', 'request_id': 'req-ec2adc95-34f2-4401-a6c8-8fb43d006e42'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.173495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 843/1688] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:35 2022] PUT /identity/v3/projects/474ba0e28d0b467db9e7221dde482308/users/c81819ed068d450583a1d66107aa85ab/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:35.183474 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.183761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.184039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.184521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] The token cache is being invalidate because project ee50ceb6e38b483ebc4443911bfd6531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:35.184912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.186094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.187982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 846/1689] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:34 2022] DELETE /identity/v3/projects/ee50ceb6e38b483ebc4443911bfd6531 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:35.192404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:35.194958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:35.213572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2adc95-34f2-4401-a6c8-8fb43d006e42 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:35.217911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] MFA Rules not processed for user `c81819ed068d450583a1d66107aa85ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:35.219187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:35.289005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:35.290002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.290394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.290858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] PATH_INFO: `/v3/users/f52a7963b21a4c8b99e24d21b5c799ed` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.292178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52a7963b21a4c8b99e24d21b5c799ed)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:35.294895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:35.298814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 844/1690] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:35.316204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:35.324227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:35.386267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b10a524-1a05-4b0a-894a-aefc2b17a0cd None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:35.394047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:35.394749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.395037 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.395358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.409811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:35.409811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:35.418955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 845/1691] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:35.520888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52a7963b21a4c8b99e24d21b5c799ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.556655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52a7963b21a4c8b99e24d21b5c799ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.557784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52a7963b21a4c8b99e24d21b5c799ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.566750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52a7963b21a4c8b99e24d21b5c799ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.567453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-83eeb12e-06eb-4a3f-9cd7-32267a1b38dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52a7963b21a4c8b99e24d21b5c799ed'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:35.628831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:35.657173 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 847/1692] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:35 2022] DELETE /identity/v3/users/f52a7963b21a4c8b99e24d21b5c799ed => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:35.676074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:35.677554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:35.696429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a04df-bc61-430f-a77e-07ea663433ab service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:35.749786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:35.755268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:35.755892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:35.756168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:35.756595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:35.759034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:35.835856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:35.837350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e2d13-7f19-400d-9753-831055768145 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:35.853252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 846/1693] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:36.733027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:36.733551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:36.733669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:36.735694 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 848/1694] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:05:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:05:36.785968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c641-b9e4-4136-9d6b-8ea407eb0b0e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:36.786548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c641-b9e4-4136-9d6b-8ea407eb0b0e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:36.786548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c701c641-b9e4-4136-9d6b-8ea407eb0b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:36.819387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c701c641-b9e4-4136-9d6b-8ea407eb0b0e None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:36.825245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c701c641-b9e4-4136-9d6b-8ea407eb0b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:36.848791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:36.850580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:37.252978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6e1449-721c-4c4a-992b-78a485b33c71 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:37.254213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 847/1695] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:05:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:37.254826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:37.254826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.254826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.254826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] PATH_INFO: `/v3/projects/b16ed6d1926940b6ade836a2a6a60124` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.255265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16ed6d1926940b6ade836a2a6a60124)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:37.255265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa876-9cbe-472d-ab1a-05de62327798 None None] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa876-9cbe-472d-ab1a-05de62327798 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32aa876-9cbe-472d-ab1a-05de62327798 None None] PATH_INFO: `/` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 848/1696] 10.222.0.60 () {54 vars in 961 bytes} [Fri Oct 21 15:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.255593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 849/1697] 10.222.0.60 () {58 vars in 1058 bytes} [Fri Oct 21 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:37.256214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2145197-f16d-4b26-8ef4-7dd56f76d31e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:37.256730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:37.256730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.256730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.256730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.257301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:37.257301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e9ef0-ada5-40b1-864b-e0198f9e2930 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:37.257301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 850/1698] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:37.316953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16ed6d1926940b6ade836a2a6a60124', 'request_id': 'req-cb16c4e8-12aa-4331-ab36-de9cc59071c5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:37.468513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16ed6d1926940b6ade836a2a6a60124', 'request_id': 'req-cb16c4e8-12aa-4331-ab36-de9cc59071c5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:37.482606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16ed6d1926940b6ade836a2a6a60124', 'request_id': 'req-cb16c4e8-12aa-4331-ab36-de9cc59071c5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:37.503609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] The token cache is being invalidate because project b16ed6d1926940b6ade836a2a6a60124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:37.505245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:37.507688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cb16c4e8-12aa-4331-ab36-de9cc59071c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:37.515469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 849/1699] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:36 2022] DELETE /identity/v3/projects/b16ed6d1926940b6ade836a2a6a60124 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:37.524298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.524298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.534797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.597554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:37.597554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:37.686118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 851/1700] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:37.687624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.687906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.694480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.715090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:37.721812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:37.758330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 850/1701] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:37.761431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:37.773697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a1c958-83be-42bd-ac6c-b330fc975b5c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:37.778598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:37.779444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.779728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.779998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.780654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:37.780955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:37.782553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:37.795039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 852/1702] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:37.799320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:37.808940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a99c17-5eba-4f18-af38-a8e38cbf840c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:37.832906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:37.833770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:37.834050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:37.834318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:37.835001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:37.836901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:37.913632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:37.941194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:37.960492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:37.961638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:37.974929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 851/1703] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:37 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:37.986040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:37.989335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac4f037-fc28-4873-810b-4f9609211c2b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.098460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23321af-9552-4b34-94a6-d5845cf29f58 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.107712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.110103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.110311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.110816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.111718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.113982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.156608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.158416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.158741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.159155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] PATH_INFO: `/v3/users/c2ec23eb23ae4c1e8ddc37e73d759733` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.163702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2ec23eb23ae4c1e8ddc37e73d759733)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.175238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.268053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 852/1704] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:37 2022] POST /identity/v3/users => generated 287 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:38.269133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:38.280738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8ed4f8-c8eb-4f4b-bc91-8d2c44f370b8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.292357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.293339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.293821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.294340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.295229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.297186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.317923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 853/1705] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:38.324749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:38.336549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1d3e3-9759-4e42-a4d9-6293e7888103 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.342592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.343451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.343949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.344386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.345667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.350814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.359901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 854/1706] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:38.364219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:38.377398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53178719-3c56-4bbd-879d-64fa49bf8956 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.382268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.385191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.385191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.385191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] PATH_INFO: `/v3/projects/780201e2794a47c294b91f39978cfbf4/users/6b59e9b0ec6e4fee99a93c58f9dcd6ba/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.401889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=780201e2794a47c294b91f39978cfbf4, user_id=6b59e9b0ec6e4fee99a93c58f9dcd6ba, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.404886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5875e9-9300-40d7-a795-e84fb716ec6b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.430591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2ec23eb23ae4c1e8ddc37e73d759733'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:38.431385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2ec23eb23ae4c1e8ddc37e73d759733'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:38.449182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2ec23eb23ae4c1e8ddc37e73d759733'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:38.533189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 855/1707] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:38 2022] PUT /identity/v3/projects/780201e2794a47c294b91f39978cfbf4/users/6b59e9b0ec6e4fee99a93c58f9dcd6ba/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:38.538299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.538716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.539134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.560514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:38.561815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:38.565512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2ec23eb23ae4c1e8ddc37e73d759733'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:38.566146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2ec23eb23ae4c1e8ddc37e73d759733'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:38.586685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 853/1708] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:37 2022] DELETE /identity/v3/users/c2ec23eb23ae4c1e8ddc37e73d759733 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:38.631521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 856/1709] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:38.641271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:38.642720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:38.714158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2773-490b-43cc-a746-d618339c400f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:38.719017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:38.719649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:38.719820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:38.720089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:38.793596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:38.795107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:38.805996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 854/1710] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:39.011871 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:39.015309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:39.049027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b202a69-2635-4db5-bc93-c7fd7db5cfca None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:39.049027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:39.049027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:39.049027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:39.049758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] PATH_INFO: `/v3/projects/ea902dc770e449bd9e53b96f753176bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:39.049758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea902dc770e449bd9e53b96f753176bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:39.050403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:39.138916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:39.219059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea902dc770e449bd9e53b96f753176bf', 'request_id': 'req-a395baed-0399-427d-a116-7a603b3c9d05'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.287585 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea902dc770e449bd9e53b96f753176bf', 'request_id': 'req-a395baed-0399-427d-a116-7a603b3c9d05'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.296663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea902dc770e449bd9e53b96f753176bf', 'request_id': 'req-a395baed-0399-427d-a116-7a603b3c9d05'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.312369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] The token cache is being invalidate because project ea902dc770e449bd9e53b96f753176bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:39.312699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.315239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.316915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 857/1711] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:39 2022] DELETE /identity/v3/projects/ea902dc770e449bd9e53b96f753176bf => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:39.322087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:39.342422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9628fc-c393-4a70-bec3-9bcf76dc02f6 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:39.418864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:39.462322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:39.463066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:39.463347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:39.463673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] PATH_INFO: `/v3/users/6746662caa574234835e595130b9802b` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:39.467048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6746662caa574234835e595130b9802b)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:39.470607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:39.830444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6746662caa574234835e595130b9802b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.831166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6746662caa574234835e595130b9802b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.846465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6746662caa574234835e595130b9802b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.944966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:39.946401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:39.953220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6746662caa574234835e595130b9802b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.953934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6746662caa574234835e595130b9802b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:39.973040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 855/1712] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:39 2022] DELETE /identity/v3/users/6746662caa574234835e595130b9802b => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:39.977192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a395baed-0399-427d-a116-7a603b3c9d05 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:40.062084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:40.063576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.064420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.064860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] PATH_INFO: `/v3/users/628f9f646c934277963412611b083515` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.066652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=628f9f646c934277963412611b083515)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:40.068902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:40.322861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '628f9f646c934277963412611b083515'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.341209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '628f9f646c934277963412611b083515'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.342093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '628f9f646c934277963412611b083515'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.349531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '628f9f646c934277963412611b083515'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.350245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4356ba25-6a81-4213-8b23-d695664d92f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '628f9f646c934277963412611b083515'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.403560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:40.407790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.435419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0272f8f-22b6-4bbb-a9ff-c23fa513a5b4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:40.440307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:40.441073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.441458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.441831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] PATH_INFO: `/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.443210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9296a1ae9d8d4bde88b599ac492dd37b)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:40.445382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 858/1713] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:39 2022] DELETE /identity/v3/users/628f9f646c934277963412611b083515 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:40.445815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:40.446776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d46ce5-330f-4352-bfeb-1f91038a844d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.447301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d46ce5-330f-4352-bfeb-1f91038a844d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.447729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d46ce5-330f-4352-bfeb-1f91038a844d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.449297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 859/1714] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:05:40.450570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0750-3228-47ff-a788-ff0569d9e970 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.451030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0750-3228-47ff-a788-ff0569d9e970 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.451523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932b0750-3228-47ff-a788-ff0569d9e970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.478762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-932b0750-3228-47ff-a788-ff0569d9e970 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:40.480375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-932b0750-3228-47ff-a788-ff0569d9e970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.548932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 860/1715] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:40.550334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.550619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.550920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.571999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] MFA Rules not processed for user `e05f107798a24fa9a0e65918076f2cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:40.573222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.612036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9296a1ae9d8d4bde88b599ac492dd37b', 'request_id': 'req-9a188a3f-08c9-4cee-8273-611213305e51'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.646673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 861/1716] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:40.651940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:40.654111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.719802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b84e310-1a65-4475-8b26-a9d31ba1c099 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:40.724484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9296a1ae9d8d4bde88b599ac492dd37b', 'request_id': 'req-9a188a3f-08c9-4cee-8273-611213305e51'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.732388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9296a1ae9d8d4bde88b599ac492dd37b', 'request_id': 'req-9a188a3f-08c9-4cee-8273-611213305e51'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.752207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] The token cache is being invalidate because project 9296a1ae9d8d4bde88b599ac492dd37b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:40.752496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.760460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:40.760460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 856/1717] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:40 2022] DELETE /identity/v3/projects/9296a1ae9d8d4bde88b599ac492dd37b => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:40.760460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:40.760460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.781892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:40.782466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.782789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.783064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.784815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.826879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a188a3f-08c9-4cee-8273-611213305e51 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:40.832706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:40.833451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.833799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.834209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.836197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.852162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:40.852791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:40.852791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:40.861069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:40.863371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040a40f-2e15-4e02-a6ed-0bfbe0bb8974 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:40.867089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 857/1718] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:40.868119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:40.869610 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.874867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 862/1719] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:40.876497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a384-93ef-4500-b77b-3d6c1b4778b9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.877965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a384-93ef-4500-b77b-3d6c1b4778b9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.878395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20a384-93ef-4500-b77b-3d6c1b4778b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.883862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b75b56e-cb2a-43de-a739-e51973c3cf37 service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:40.889546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:40.890682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.891127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.891720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] PATH_INFO: `/v3/users/c82ec83bb124455693573e6c68ff12fb` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.901271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c82ec83bb124455693573e6c68ff12fb)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:40.908880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7e20a384-93ef-4500-b77b-3d6c1b4778b9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:40.909939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:40.910274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7e20a384-93ef-4500-b77b-3d6c1b4778b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.928861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 863/1720] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:40.937673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:40.937963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:40.938295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:40.971652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:40.972863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:40.998796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 864/1721] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:40.999945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.010128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568e6a6-e947-46bc-9b81-4f91e5638ba1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.015274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.015928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.016215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.016519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.017170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:41.017489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.019159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238917ed-8047-4560-a78a-f537c20fca71 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.031130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 865/1722] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:41.032383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.032667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.033116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.065844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] MFA Rules not processed for user `6b59e9b0ec6e4fee99a93c58f9dcd6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:41.067121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.145783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 866/1723] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:41.146857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.162411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095a6fc5-4f0d-4396-9047-4647efcd41e1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.169835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c82ec83bb124455693573e6c68ff12fb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.170727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c82ec83bb124455693573e6c68ff12fb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.170971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.171601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.171877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.172385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.173187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.175193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.186385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c82ec83bb124455693573e6c68ff12fb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.239171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:41.273151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c82ec83bb124455693573e6c68ff12fb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.273771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c82ec83bb124455693573e6c68ff12fb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.280892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 867/1724] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:41 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:41.281625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.283292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.288532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 858/1725] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:40 2022] DELETE /identity/v3/users/c82ec83bb124455693573e6c68ff12fb => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:41.289517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.291092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.348627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfd27-b738-46f5-9ff1-0f9ca5cbdd66 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.373513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddaed2-9077-4223-b7da-d30e63926bf6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.380585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.381451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.381733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.382001 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.382672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.384975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.408341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.409101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.409390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.409661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.411636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.468302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 859/1726] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:41 2022] POST /identity/v3/users => generated 286 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:41.469204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.470894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.485156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227b9a96-d444-43f6-9912-4152b96a85df admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.488054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.490655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.491108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.491331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.491613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.491891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] PATH_INFO: `/v3/projects/665f96c7ad5344b2a7f735c5fcc8d650` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.493038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665f96c7ad5344b2a7f735c5fcc8d650)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.495121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.502823 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 868/1727] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:41.505145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.515759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89db39f-3cf3-4b35-8907-b95212b4287f service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.520760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.521474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.521756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.522033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.523126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.524375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.535139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 869/1728] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:41.539127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.550997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a195973c-cc60-486e-b201-4c11a43b183c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.557735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.558658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.558658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.559674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.560564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.562028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.572389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 870/1729] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:41.578501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.588403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad64839-48db-4e00-9b43-bcdf1b02a0ff admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.594154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.594862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.595147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.595423 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] PATH_INFO: `/v3/projects/ef29f8dbc50c486cbaf73c7394324973/users/91bf0a6b413a4ff298bc9abb4161e300/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.610876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef29f8dbc50c486cbaf73c7394324973, user_id=91bf0a6b413a4ff298bc9abb4161e300, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.613570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.732262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 871/1730] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:41 2022] PUT /identity/v3/projects/ef29f8dbc50c486cbaf73c7394324973/users/91bf0a6b413a4ff298bc9abb4161e300/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:41.741436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665f96c7ad5344b2a7f735c5fcc8d650', 'request_id': 'req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.743008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:41.744773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:41.826238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe39445-c535-4485-9fbc-2f95d0ff75a1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:41.873975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665f96c7ad5344b2a7f735c5fcc8d650', 'request_id': 'req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.882961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665f96c7ad5344b2a7f735c5fcc8d650', 'request_id': 'req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.894713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:41.894713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.894713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.894713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] PATH_INFO: `/v3/projects/6406e245a2fd4700aebba95795a18632` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.895174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6406e245a2fd4700aebba95795a18632)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:41.895473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:41.908765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] The token cache is being invalidate because project 665f96c7ad5344b2a7f735c5fcc8d650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:41.909335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.910033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-bfa6ca86-9a75-41a5-b5a5-778c8e9bf2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:41.912363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 860/1731] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:41 2022] DELETE /identity/v3/projects/665f96c7ad5344b2a7f735c5fcc8d650 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:41.913916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:41.917795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:41.917795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:41.946048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] MFA Rules not processed for user `91bf0a6b413a4ff298bc9abb4161e300`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:41.947427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.022369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 861/1732] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:42.023251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.026563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.036516 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6406e245a2fd4700aebba95795a18632', 'request_id': 'req-6eb261af-4759-4154-8473-a92709235da6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.049411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f39d93c-0952-4501-bdbc-7aead68ae9ab None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.076906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:42.107858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6406e245a2fd4700aebba95795a18632', 'request_id': 'req-6eb261af-4759-4154-8473-a92709235da6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.109358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.110011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.110296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.110571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] PATH_INFO: `/v3/users/c81819ed068d450583a1d66107aa85ab` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.111798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81819ed068d450583a1d66107aa85ab)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.113729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.115087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6406e245a2fd4700aebba95795a18632', 'request_id': 'req-6eb261af-4759-4154-8473-a92709235da6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.125989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] The token cache is being invalidate because project 6406e245a2fd4700aebba95795a18632 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:42.126281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.126960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.128742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 872/1733] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:41 2022] DELETE /identity/v3/projects/6406e245a2fd4700aebba95795a18632 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:42.129378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.131142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.192761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb261af-4759-4154-8473-a92709235da6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.196416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.197146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.197632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.198088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.199795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:42.201264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.267405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.268982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d74b76e-7994-4cd1-98c2-eb4aaa0f1d01 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.278013 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 873/1734] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:42.283127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.283127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.283127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.311965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:42.313260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81819ed068d450583a1d66107aa85ab'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.313396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.314074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81819ed068d450583a1d66107aa85ab'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.321452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 874/1735] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:42.322244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.326789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81819ed068d450583a1d66107aa85ab'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.328626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.344348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddf33e0-a967-4f48-8eca-9fac29ea77de None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.349200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.349884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.350156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.350424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] PATH_INFO: `/v3/users/91bf0a6b413a4ff298bc9abb4161e300` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.351718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91bf0a6b413a4ff298bc9abb4161e300)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.353660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.376114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81819ed068d450583a1d66107aa85ab'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.377003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-505b160f-60d8-4d43-a9ed-cf15e466b0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81819ed068d450583a1d66107aa85ab'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.394050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 862/1736] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:42 2022] DELETE /identity/v3/users/c81819ed068d450583a1d66107aa85ab => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:42.395638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.395939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.396270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.423304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:42.424531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.492018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 863/1737] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:42.496364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.505318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf19e8-db0e-4478-875c-a9d3b270409b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.509819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.510403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.510712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.511036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.511673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:42.511969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.514210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.525690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 864/1738] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:05:42.530358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.540388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadcd9f-9a7d-4ea7-a2df-dd1997f20934 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.545425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.546018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.546297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.546567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.547227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.549222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.602998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91bf0a6b413a4ff298bc9abb4161e300'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.615298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91bf0a6b413a4ff298bc9abb4161e300'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.616049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91bf0a6b413a4ff298bc9abb4161e300'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.623765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91bf0a6b413a4ff298bc9abb4161e300'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.624385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-94467d8e-25ff-4d20-a31b-068a3b82a277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91bf0a6b413a4ff298bc9abb4161e300'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:42.679257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 865/1739] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:42 2022] POST /identity/v3/projects => generated 377 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:42.683029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.685710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.705560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 875/1740] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:42 2022] DELETE /identity/v3/users/91bf0a6b413a4ff298bc9abb4161e300 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:42.709981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.710272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.710548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.733367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:42.734608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.760217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fceaa27-4523-430d-9574-7697da10ed50 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.806729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 876/1741] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:42.822320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.824399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.840013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.840700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.841563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.841921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.842777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.844785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.888946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1adeebd-06fc-4ea8-9c9e-de77d8a01450 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.892766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.893420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.893695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.894071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.904250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.905810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.915361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 877/1742] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:42.937488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:42.939257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:42.959720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c9bd35-f4cd-49ff-a836-f33c0ebd8e77 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:42.970699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:42.970699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:42.970699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:42.970699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] PATH_INFO: `/v3/projects/474ba0e28d0b467db9e7221dde482308` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:42.971391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474ba0e28d0b467db9e7221dde482308)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:42.971391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:42.987817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 866/1743] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:42 2022] POST /identity/v3/users => generated 289 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:42.988237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.005282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2178ef-48c6-48f3-87e1-af695d44e4b4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.011356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.012370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.012588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.013236 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.014806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.016508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.028090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 867/1744] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:43.032916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.043487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6036b879-45dd-4e00-9c04-353b61d87ffc admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.049873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.050489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.050804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.057870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.057870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.057870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.070098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 868/1745] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:43.074502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.084902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9923bfb4-6707-4a31-843c-fcaa82202b25 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.089520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.090167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.092399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.092399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] PATH_INFO: `/v3/projects/0025ac2cdc0f45dab742a02486307716/users/3418ee58bd51418d9594b98ed594f506/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.107742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0025ac2cdc0f45dab742a02486307716, user_id=3418ee58bd51418d9594b98ed594f506, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.110544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3bb38-903f-4213-b96b-8e8eb7d1a89e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.216587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474ba0e28d0b467db9e7221dde482308', 'request_id': 'req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.217472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 869/1746] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:43 2022] PUT /identity/v3/projects/0025ac2cdc0f45dab742a02486307716/users/3418ee58bd51418d9594b98ed594f506/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:43.218013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.218235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.218512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.239532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.240747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.307610 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 870/1747] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:43.314896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474ba0e28d0b467db9e7221dde482308', 'request_id': 'req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.315800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.327023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474ba0e28d0b467db9e7221dde482308', 'request_id': 'req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.335235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] The token cache is being invalidate because project 474ba0e28d0b467db9e7221dde482308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:43.335601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.336260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7dc92c8c-6456-4ff9-8b6b-a2a9b72baf45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.337939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 878/1748] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:42 2022] DELETE /identity/v3/projects/474ba0e28d0b467db9e7221dde482308 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:43.343277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343435-c9b7-44d0-8868-d9c310129ef7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.343559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343435-c9b7-44d0-8868-d9c310129ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.343929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343435-c9b7-44d0-8868-d9c310129ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.378369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-44343435-c9b7-44d0-8868-d9c310129ef7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.378369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-44343435-c9b7-44d0-8868-d9c310129ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.386091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 879/1749] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:43.386248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9a4f14-31b3-4c24-8bd4-58f68b06f85b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.388852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:43.391288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.391576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.391852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.393229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.393913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.394201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.394480 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.396308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.408856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.410316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f334c6f-5c0f-4f0a-b493-015c131fa4b1 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.411773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] MFA Rules not processed for user `6b59e9b0ec6e4fee99a93c58f9dcd6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.413004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.430969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 871/1750] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:43.432809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.432999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.433328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.470882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.472368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.477460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 880/1751] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:43.479647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.480611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 872/1752] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:43.481005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.484830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.498770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cad9ec-93a7-4a8c-9c85-97b04dcbc4b4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.502605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.503004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6046194b-b6e7-4a07-a2ad-654d8b0a7404 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.503420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.503768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.503981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] PATH_INFO: `/v3/projects/ef29f8dbc50c486cbaf73c7394324973` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.505125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef29f8dbc50c486cbaf73c7394324973)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.507157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.508896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.511481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.511983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.512358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.513850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:43.514282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.515986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.533129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 873/1753] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:43.537691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.551114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6074042f-d89e-4a52-aaab-a290a2b36a10 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.556027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.556624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.556902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.557297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.557925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.560125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.707379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef29f8dbc50c486cbaf73c7394324973', 'request_id': 'req-9013b600-deed-4fc5-9130-1410f7f354e9'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.771115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 874/1754] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/projects => generated 369 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:43.772413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.787715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef29f8dbc50c486cbaf73c7394324973', 'request_id': 'req-9013b600-deed-4fc5-9130-1410f7f354e9'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.799153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef29f8dbc50c486cbaf73c7394324973', 'request_id': 'req-9013b600-deed-4fc5-9130-1410f7f354e9'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.814010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] The token cache is being invalidate because project ef29f8dbc50c486cbaf73c7394324973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:43.814566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.815467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9013b600-deed-4fc5-9130-1410f7f354e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:43.817376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 881/1755] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:43 2022] DELETE /identity/v3/projects/ef29f8dbc50c486cbaf73c7394324973 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:43.822603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af5204-3a34-490f-82cc-44b64688d093 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.822945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af5204-3a34-490f-82cc-44b64688d093 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.823228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af5204-3a34-490f-82cc-44b64688d093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.847233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d68343-4cc2-41c7-b8c8-be78e721411f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.852489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.853173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.853458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.853737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.854402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.857132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.868140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a1af5204-3a34-490f-82cc-44b64688d093 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.870709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a1af5204-3a34-490f-82cc-44b64688d093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.892339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 882/1756] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:43.895562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.895926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.896273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.918123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:43.920115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.928737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 883/1757] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:43.933009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.943116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d6f492-61b1-4d90-b8fe-1f5301765155 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.948100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.949220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.949503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.949779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:43.950473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:43.950899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:43.957999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:43.965275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 875/1758] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:43.966979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.968857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:43.976286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 884/1759] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:43.983315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:43.993300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0350e43-19c5-4bc3-8790-89660bb09a7c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.996464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ed987e-5c13-402d-8ade-d4014b2dccbf admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:43.998472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:43.999220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:43.999577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:43.999921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.000661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.002343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.002486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.003252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.003672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.004210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.005018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.006935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.017823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 876/1760] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:44.022554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.033955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479e39d-c3f9-4001-b1bc-2129664c3336 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.039044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.040100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.040383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.040654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.041293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.043082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.060612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 877/1761] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:44.064805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.074727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f82c4b-a61c-47e5-b48a-f48f8b6ebd07 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.080717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.081432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.081766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.082040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] PATH_INFO: `/v3/projects/f6c127c93b07401a8d8f4c1364bdb688/users/50f1db5c531142c1a010fa09f95b0b84/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.100302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6c127c93b07401a8d8f4c1364bdb688, user_id=50f1db5c531142c1a010fa09f95b0b84, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.104083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023d31a-b3bb-4ca0-a2dc-b9bf89cfda84 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.123852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 885/1762] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:43 2022] POST /identity/v3/projects => generated 363 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:44.125087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.193851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 878/1763] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:44 2022] PUT /identity/v3/projects/f6c127c93b07401a8d8f4c1364bdb688/users/50f1db5c531142c1a010fa09f95b0b84/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:44.206030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.206030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.206030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.213544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677a0f9-ae99-4c64-b00f-8b9f2eb2564f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.238503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] MFA Rules not processed for user `50f1db5c531142c1a010fa09f95b0b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:44.239349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:44.269688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:44.290137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.291324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.291324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.291324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.292001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.293935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.326694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 879/1764] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:44.327801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.339885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4645efa1-233f-4ffb-95ec-bb0d4fbb3e3c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.346274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.347071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.347268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.347498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.348141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.350052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.473146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 886/1765] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:44 2022] POST /identity/v3/users => generated 282 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:44.477306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.487619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7447f48-495c-4cac-a5e2-48b3ec8d8ca8 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.492348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.492939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.493224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.493502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.494140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.507210 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 880/1766] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:44 2022] POST /identity/v3/projects => generated 369 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:44.510829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.515388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.548762 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 887/1767] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:44.552376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.573134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc002e-56fb-4ea3-b042-ea7e037f727e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.606086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b77d4-bad9-4b26-a26e-0e39448b7fea admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.613264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.614538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.615085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.615534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.616025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:44.630787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.630787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.687663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.688539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.688688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.689033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.689869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.691766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.705774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 888/1768] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:44.709047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.720253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03faeac3-6c6c-414f-9dea-5879e3388ddf admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.726179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.727339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.727613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.727948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] PATH_INFO: `/v3/projects/b3f8918528774d3c89b883f4cc7eb0b5/users/9b160e28dc6244c7aaf53d1fa165efc2/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.745963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3f8918528774d3c89b883f4cc7eb0b5, user_id=9b160e28dc6244c7aaf53d1fa165efc2, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.749580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048a2ba-e233-4402-9d29-03549956b1b7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.760842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 881/1769] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:44 2022] POST /identity/v3/users => generated 291 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:44.765144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.775140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19704154-84b8-49c0-b408-a61cf42bd607 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.780031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.780624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.780998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.781271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.781908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.783528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.796931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 882/1770] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:44.800708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.811740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c6988-a10e-4645-a966-52519f0f8d3a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:44.816732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:44.817397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.819357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.819357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] PATH_INFO: `/v3/projects/95881504ae6f4c38bd7da13675a8b375/users/4cd5218a94944f598bfc6933f7cb6343/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.833738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95881504ae6f4c38bd7da13675a8b375, user_id=4cd5218a94944f598bfc6933f7cb6343, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:44.836442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:44.867609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 889/1771] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:44 2022] PUT /identity/v3/projects/b3f8918528774d3c89b883f4cc7eb0b5/users/9b160e28dc6244c7aaf53d1fa165efc2/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:44.879896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:44.880284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:44.880635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:44.910086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] MFA Rules not processed for user `9b160e28dc6244c7aaf53d1fa165efc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:44.911438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:44.925298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 883/1772] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:44 2022] PUT /identity/v3/projects/95881504ae6f4c38bd7da13675a8b375/users/4cd5218a94944f598bfc6933f7cb6343/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:44.926928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:44.994743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 890/1773] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:45.012616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:45.012616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:45.020219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61963e86-348d-4b63-9650-be8b6801d979 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:45.025021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:45.025754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:45.026033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:45.026312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:45.026982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:45.028751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:45.039050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 884/1774] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:05:45.043436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:45.054278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1474afb-1bc8-474b-b803-43f1b3dadc95 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:45.059086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:45.060920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:45.061453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:45.061733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] PATH_INFO: `/v3/projects/95881504ae6f4c38bd7da13675a8b375/users/4cd5218a94944f598bfc6933f7cb6343/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:45.063717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95881504ae6f4c38bd7da13675a8b375, user_id=4cd5218a94944f598bfc6933f7cb6343, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:45.066341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c43ce-1d13-4cfb-bcdc-f8562c076eb7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:45.070401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844481c-7668-49c1-8bad-42a71c304707 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:45.077876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 885/1775] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:45 2022] PUT /identity/v3/projects/95881504ae6f4c38bd7da13675a8b375/users/4cd5218a94944f598bfc6933f7cb6343/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:05:45.088631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:45.088942 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:45.089185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:45.119329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] MFA Rules not processed for user `4cd5218a94944f598bfc6933f7cb6343`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:45.120549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:45.133225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:45.134481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:45.135441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:45.135913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:45.210164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:45.211134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 886/1776] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:45.211648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:45.221313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 891/1777] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:45.222604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:45.224012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:45.237161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701614b5-a1fa-42cd-a76c-d6474ba9986e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:45.240820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:45.241434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:45.241715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:45.241991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:45.251373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:45.252730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05e1aa8-5d8a-43e6-8864-020846fb5188 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:45.262517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 887/1778] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:46.844606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:47.033460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:47.673552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:49.058477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:49.153354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc046f4-3b35-4e81-b1d9-0b136b832033 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:49.160317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:49.168820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:49.168820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:49.168820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] PATH_INFO: `/v3/users/50f1db5c531142c1a010fa09f95b0b84` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:49.168820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f1db5c531142c1a010fa09f95b0b84)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:49.168820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:49.285800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:49.462914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f1db5c531142c1a010fa09f95b0b84'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:49.492478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f1db5c531142c1a010fa09f95b0b84'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:49.493330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f1db5c531142c1a010fa09f95b0b84'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:49.502276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f1db5c531142c1a010fa09f95b0b84'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:49.503459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f1db5c531142c1a010fa09f95b0b84'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:49.729004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:49.733061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 892/1779] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:49 2022] DELETE /identity/v3/users/50f1db5c531142c1a010fa09f95b0b84 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:49.737809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:49.738100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:49.738459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:49.758593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:49.760290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:49.868968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 888/1780] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:49.869377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:49.870883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:49.968671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48570b-e618-4452-a07f-ea81aebadf2c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:49.974231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:49.975623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:49.975623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:49.975623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:50.004128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:50.005743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:50.045427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 893/1781] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:05:50.681013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:50.682602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:50.702796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1033836-d06f-4c56-ac35-c4ca19197bb1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:50.714794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:50.715700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:50.716078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:50.716943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] PATH_INFO: `/v3/projects/f6c127c93b07401a8d8f4c1364bdb688` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:50.719458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6c127c93b07401a8d8f4c1364bdb688)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:50.728853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:51.018024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6c127c93b07401a8d8f4c1364bdb688', 'request_id': 'req-2707029d-b123-414c-a69f-b8b2dfd89190'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.172126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6c127c93b07401a8d8f4c1364bdb688', 'request_id': 'req-2707029d-b123-414c-a69f-b8b2dfd89190'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.181796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6c127c93b07401a8d8f4c1364bdb688', 'request_id': 'req-2707029d-b123-414c-a69f-b8b2dfd89190'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.194975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] The token cache is being invalidate because project f6c127c93b07401a8d8f4c1364bdb688 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:51.195270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.196081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2707029d-b123-414c-a69f-b8b2dfd89190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.197689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 889/1782] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:50 2022] DELETE /identity/v3/projects/f6c127c93b07401a8d8f4c1364bdb688 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:51.202038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:51.216772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff7a63-ccf6-4443-9d8d-04c378df5e33 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:51.291850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:51.292705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:51.293006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:51.293565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] PATH_INFO: `/v3/users/4cd5218a94944f598bfc6933f7cb6343` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:51.295811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cd5218a94944f598bfc6933f7cb6343)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:51.298074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:51.623408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cd5218a94944f598bfc6933f7cb6343'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.649998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cd5218a94944f598bfc6933f7cb6343'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.651129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cd5218a94944f598bfc6933f7cb6343'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.660342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cd5218a94944f598bfc6933f7cb6343'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.661430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cd5218a94944f598bfc6933f7cb6343'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:51.742116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 894/1783] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:05:51 2022] DELETE /identity/v3/users/4cd5218a94944f598bfc6933f7cb6343 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:52.061684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.062200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.062428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.131056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] MFA Rules not processed for user `3418ee58bd51418d9594b98ed594f506`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:52.132387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:52.240477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 890/1784] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:52.255570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:52.257206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:52.307287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:52.309105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:52.367726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248a4767-fc83-4606-95a7-2e3cc2fab824 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:52.374540 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:52.387115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.387115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.387115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.387115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:52.387115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f62cf5e-b1e9-4eea-9619-d9f722b868b5 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:52.397648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 895/1785] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:52.431472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fa06a-bdaa-4834-ad96-5fdeaeed0561 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:52.437349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:52.438021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.438301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.438575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] PATH_INFO: `/v3/projects/95881504ae6f4c38bd7da13675a8b375` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.439767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95881504ae6f4c38bd7da13675a8b375)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:52.441672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:52.446812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:52.576129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95881504ae6f4c38bd7da13675a8b375', 'request_id': 'req-5b638107-f75e-448d-addb-2023194fb740'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:52.679251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95881504ae6f4c38bd7da13675a8b375', 'request_id': 'req-5b638107-f75e-448d-addb-2023194fb740'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:52.686584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95881504ae6f4c38bd7da13675a8b375', 'request_id': 'req-5b638107-f75e-448d-addb-2023194fb740'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:52.701304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] The token cache is being invalidate because project 95881504ae6f4c38bd7da13675a8b375 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:05:52.702534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:52.703388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-5b638107-f75e-448d-addb-2023194fb740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:05:52.705211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 891/1786] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:05:52 2022] DELETE /identity/v3/projects/95881504ae6f4c38bd7da13675a8b375 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:52.710181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.710474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.711720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.739431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:52.740979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:52.776501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:52.827350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 896/1787] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:05:52.833388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.833783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.834070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.881735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:52.884159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:52.908324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 892/1788] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:52.913016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:52.926489 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4058783-0e7d-4aa7-b8cd-abf48ccc66f3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:52.949164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:52.954565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:52.985997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 897/1789] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:05:52.986900 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:53.007930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e3be1b-1a2b-4542-afcb-d795353e9e99 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:53.007930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:53.007930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.007930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.008199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.008199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:53.025735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:53.168743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:53.179758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 893/1790] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:52 2022] POST /identity/v3/projects => generated 381 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:05:53.181311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89ed8-8237-42ae-99d9-381f9fbba754 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:53.262406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:53.264427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.264984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.265373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.266025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:53.268268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:53.382443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 898/1791] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:53 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:53.385050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:53.398548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc4e068-667b-456b-9899-0debf2a5608e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:53.419389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:53.423039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.423494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.423963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.424629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:53.426511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:53.453601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 894/1792] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:53.457789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:53.471312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3604ae-7887-484b-b390-fd4fb2d09d01 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:53.483774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:53.484691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.485163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.485710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.486558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:53.488741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4fc8f-6d40-4d1b-89ec-2f1c477ba437 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:53.510041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:53.519993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 899/1793] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:53.526198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab4b23-a490-4419-b05c-d7b32cef140b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:53.534405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:53.536205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.536835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.537785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] PATH_INFO: `/v3/projects/c6b397c8b3fb4f8491f3cbf4facf9fa7/users/012a6471531a40899be0f16082d35665/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.561340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b397c8b3fb4f8491f3cbf4facf9fa7, user_id=012a6471531a40899be0f16082d35665, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:53.566261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:53.675601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 895/1794] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:53 2022] PUT /identity/v3/projects/c6b397c8b3fb4f8491f3cbf4facf9fa7/users/012a6471531a40899be0f16082d35665/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:53.686136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:53.686965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:53.687476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:53.766176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] MFA Rules not processed for user `012a6471531a40899be0f16082d35665`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:53.773868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:53.893608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 900/1795] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:53.898588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.009278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f678b9f-8cd5-4123-93f3-8a5c9102f638 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.018093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.019348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.019642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.020328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.021420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.026989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.163680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 896/1796] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:05:53 2022] POST /identity/v3/projects => generated 383 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:54.166689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.182427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104c1b2b-7090-4518-ba36-131f58366dfb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.256869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.257927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.257927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.258119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.258826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.261008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.371267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 901/1797] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:05:54 2022] POST /identity/v3/users => generated 298 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:05:54.375526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.387223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f9877b-1ad0-4c03-bf9d-b38972693564 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.393427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.394331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.396107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.396107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.396107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.397593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.409341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 897/1798] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:54.413542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.425887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbd62e-d15c-4b90-9d2f-f02c68f723d1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.432556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.433237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.433521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.433799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] PATH_INFO: `/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e/users/a340f29c59f345f8af3a303f6a7ae081/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.453786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca4cd528b1741df8b6f6f93fb9c6d7e, user_id=a340f29c59f345f8af3a303f6a7ae081, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.456586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.611786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 902/1799] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:54 2022] PUT /identity/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e/users/a340f29c59f345f8af3a303f6a7ae081/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:54.615596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.626877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b322c-02f8-424c-a6dc-814d47a67b8e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.719670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.720306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.720868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.721321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.721998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.724076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b193be5-3a99-4a01-aa60-00339e15857e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.740868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 898/1800] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:05:54.746308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.771222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f0e31-20fe-455a-ae14-3091f4e4584a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:54.776756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:54.777557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.777927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.778401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] PATH_INFO: `/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e/users/a340f29c59f345f8af3a303f6a7ae081/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.783389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca4cd528b1741df8b6f6f93fb9c6d7e, user_id=a340f29c59f345f8af3a303f6a7ae081, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:54.791251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:54.813569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 903/1801] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:05:54 2022] PUT /identity/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e/users/a340f29c59f345f8af3a303f6a7ae081/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:05:54.826259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:54.826557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:54.826867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:54.861712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] MFA Rules not processed for user `a340f29c59f345f8af3a303f6a7ae081`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:54.863105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:54.969532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 899/1802] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:54.986675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:54.991190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:55.004995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:55.083678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd86c3-a0e4-46e9-b3bb-ca7cb12d42cb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:55.088357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:55.089504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:55.090245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:55.091454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:55.115469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:55.117384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff605b-f779-48ad-aada-805f09a79531 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:55.132494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 904/1803] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:55.271989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:05:55.279805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:55.288638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37f84a8-a342-4048-b5ec-7fe176f579c2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:05:55.293198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:05:55.299232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:55.302832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:55.303669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:55.396120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:05:55.397638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:05:55.409476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 900/1804] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:05:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:05:55.834754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501761-4f83-4973-86d1-64b3ab0afc12 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:05:55.835657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501761-4f83-4973-86d1-64b3ab0afc12 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:05:55.835999 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18501761-4f83-4973-86d1-64b3ab0afc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:05:55.858568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-18501761-4f83-4973-86d1-64b3ab0afc12 None None] MFA Rules not processed for user `3418ee58bd51418d9594b98ed594f506`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:05:55.859873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-18501761-4f83-4973-86d1-64b3ab0afc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:05:55.925067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 905/1805] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:05:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:05:56.975933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:57.645784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:58.296177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:58.576481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:58.672965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:58.993304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:59.176623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:59.472864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:05:59.547656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:00.004567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:00.072962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:00.484240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:02.819233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:02.900890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:02.909174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb0ed7-b5aa-4d0a-bfd8-8a9dced26483 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:02.914095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:02.915144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:02.915435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:02.916401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] PATH_INFO: `/v3/users/012a6471531a40899be0f16082d35665` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:02.917270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=012a6471531a40899be0f16082d35665)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:02.919613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:03.132699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012a6471531a40899be0f16082d35665'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:03.133557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012a6471531a40899be0f16082d35665'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:03.165896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012a6471531a40899be0f16082d35665'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:03.244383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012a6471531a40899be0f16082d35665'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:03.245170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012a6471531a40899be0f16082d35665'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:03.260545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 901/1806] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:02 2022] DELETE /identity/v3/users/012a6471531a40899be0f16082d35665 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:03.265865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:03.266752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:03.267298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:03.292373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:03.294183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:03.365883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 906/1807] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:03.378343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:03.380184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:03.452380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02c3dea-e064-4e26-b08c-c2b14489ff42 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:03.458837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:03.459708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:03.460156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:03.460623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:03.484874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:03.487061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:03.503017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 902/1808] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:04.229430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:04.231429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:04.244789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b962a7e-c267-4f20-8c5e-36b706b27f14 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:04.250263 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:04.250945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:04.251227 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:04.251498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] PATH_INFO: `/v3/projects/c6b397c8b3fb4f8491f3cbf4facf9fa7` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:04.253109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b397c8b3fb4f8491f3cbf4facf9fa7)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:04.256189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:04.385865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:04.401774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b397c8b3fb4f8491f3cbf4facf9fa7', 'request_id': 'req-0d4bceab-d5d4-4081-9ff3-5f78314e4502'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.506863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b397c8b3fb4f8491f3cbf4facf9fa7', 'request_id': 'req-0d4bceab-d5d4-4081-9ff3-5f78314e4502'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.514374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b397c8b3fb4f8491f3cbf4facf9fa7', 'request_id': 'req-0d4bceab-d5d4-4081-9ff3-5f78314e4502'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.524993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] The token cache is being invalidate because project c6b397c8b3fb4f8491f3cbf4facf9fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:04.525278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.525920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.527422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 907/1809] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:04 2022] DELETE /identity/v3/projects/c6b397c8b3fb4f8491f3cbf4facf9fa7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:04.531934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:04.542052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1400660-009c-45f5-8958-7f8a81fb6ade service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:04.599893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:04.600623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:04.600923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:04.601227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] PATH_INFO: `/v3/users/a340f29c59f345f8af3a303f6a7ae081` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:04.602410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] RBAC: Authorizing `identity:delete_user(user_id=a340f29c59f345f8af3a303f6a7ae081)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:04.604509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:04.626265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:04.826790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a340f29c59f345f8af3a303f6a7ae081'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.832251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a340f29c59f345f8af3a303f6a7ae081'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.841166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a340f29c59f345f8af3a303f6a7ae081'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.924689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a340f29c59f345f8af3a303f6a7ae081'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.930607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f184ef84-a4da-48d7-88b2-13b2b554cfda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a340f29c59f345f8af3a303f6a7ae081'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:04.958118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 903/1810] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:04 2022] DELETE /identity/v3/users/a340f29c59f345f8af3a303f6a7ae081 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:05.431608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:05.433321 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:05.447900 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bceab-d5d4-4081-9ff3-5f78314e4502 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:05.507736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:05.508304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:05.508587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:05.508861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] PATH_INFO: `/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:05.510048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca4cd528b1741df8b6f6f93fb9c6d7e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:05.512015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:05.632761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca4cd528b1741df8b6f6f93fb9c6d7e', 'request_id': 'req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:05.729512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca4cd528b1741df8b6f6f93fb9c6d7e', 'request_id': 'req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:05.742849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca4cd528b1741df8b6f6f93fb9c6d7e', 'request_id': 'req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:05.747640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] The token cache is being invalidate because project 7ca4cd528b1741df8b6f6f93fb9c6d7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:05.747924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:05.748573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-4663e5e1-1bdc-486e-9f3a-f5b7b95a10bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:05.750052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 908/1811] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:05 2022] DELETE /identity/v3/projects/7ca4cd528b1741df8b6f6f93fb9c6d7e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:05.754946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:05.755418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:05.755531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:05.776583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:05.777764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:05.845200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 904/1812] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:05.852973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:05.855388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:05.855388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:05.875350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:05.876554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:05.884749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 909/1813] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:05.890260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:05.899945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079341b-4c03-49a1-a844-e75a68caadde None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:05.905323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:05.906998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:05.906998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:05.906998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:05.907248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:05.907487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:05.909128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:05.921574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 905/1814] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:05.925948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:05.936269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bd86dd-9382-473e-a553-b10ba39d13e7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:05.941156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:05.941741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:05.942016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:05.942285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:05.943037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:05.947296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.038302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 910/1815] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:05 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:06.042770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.054216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbbb23-65c3-4a6c-afb4-d58a376343eb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.134234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.135144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.135532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.135909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.136665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.139554 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.223318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 906/1816] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:06 2022] POST /identity/v3/users => generated 284 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:06.228135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.240708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2949b4-71d3-40bc-aa38-db3867b1a27b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.247141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.247773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.248057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.248336 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.248998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.250607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.256974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.259180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:06.263730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 911/1817] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:06.272366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.287211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf2658-bc03-41ad-b7ba-c02de370fd90 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.290575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4069df37-6297-4b65-b533-378c8386112c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.292937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.293672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.294012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.294793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.295087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.296811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.299273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:06.313503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 912/1818] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:06.313711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.313711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.313711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.313711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] PATH_INFO: `/v3/users/bfd7118cba6a46f79f520bfe197b53dc` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.314645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.314764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd7118cba6a46f79f520bfe197b53dc)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.316734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.327769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:06.327958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f01de-9247-48a8-981a-63c856fe16d4 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.332685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.333352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.334064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.334064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] PATH_INFO: `/v3/projects/e60eb360cc404961a14cb18e1e2a4c6d/users/10d81d81253b4c0d98088ce2e96ee337/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.349379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60eb360cc404961a14cb18e1e2a4c6d, user_id=10d81d81253b4c0d98088ce2e96ee337, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.352163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54303-b912-4748-a466-be40e322c591 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.427199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 913/1819] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:06 2022] PUT /identity/v3/projects/e60eb360cc404961a14cb18e1e2a4c6d/users/10d81d81253b4c0d98088ce2e96ee337/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:06.438350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41266-7326-4236-a604-79a086a4b832 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.438945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41266-7326-4236-a604-79a086a4b832 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.438945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba41266-7326-4236-a604-79a086a4b832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.467387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-dba41266-7326-4236-a604-79a086a4b832 None None] MFA Rules not processed for user `10d81d81253b4c0d98088ce2e96ee337`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:06.468591 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dba41266-7326-4236-a604-79a086a4b832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:06.518279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd7118cba6a46f79f520bfe197b53dc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:06.518972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd7118cba6a46f79f520bfe197b53dc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:06.531777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd7118cba6a46f79f520bfe197b53dc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:06.536597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 914/1820] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:06.541623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41266-7326-4236-a604-79a086a4b832 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.587373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd7118cba6a46f79f520bfe197b53dc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:06.588018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-24e30e95-2fff-4022-a283-ca131db2a88f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd7118cba6a46f79f520bfe197b53dc'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:06.602444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 907/1821] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:06 2022] DELETE /identity/v3/users/bfd7118cba6a46f79f520bfe197b53dc => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:06.603733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba41266-7326-4236-a604-79a086a4b832 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.608220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.608547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.608829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.628910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:06.630237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:06.666033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.666659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.666946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.667218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.667851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.670481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.685408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:06.694251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 908/1822] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:06.704088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.705495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:06.764626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc610a59-fad4-44f8-9606-e2dd3b57f813 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.768548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.769372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.769808 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.775883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.782078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.783606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.783926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 915/1823] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:06 2022] POST /identity/v3/projects => generated 367 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:06.787889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.789527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:06.797395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 909/1824] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:06.883397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3e7c15-0d13-4606-bd65-cccc972e3da1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.899304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.899935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.900214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.900481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:06.901114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:06.904208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:06.967283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 916/1825] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:06 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:06.970929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:06.984091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676feba9-0ff8-4d86-a46e-7eff1f010a48 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:06.990841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:06.991545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:06.991884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:06.992227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.003690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.007306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.021599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 910/1826] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:07.025779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.038135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76e2f6-5859-48f6-992d-7219db6c26fa admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.043790 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.044479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.044738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.045056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] PATH_INFO: `/v3/projects/a8c25e2c7c7544d78b878797ee85ec01/users/032f54099115403f8c77d7bc5672de66/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.062085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c25e2c7c7544d78b878797ee85ec01, user_id=032f54099115403f8c77d7bc5672de66, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.065495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.181338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 917/1827] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:07 2022] PUT /identity/v3/projects/a8c25e2c7c7544d78b878797ee85ec01/users/032f54099115403f8c77d7bc5672de66/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:07.185446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.195588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ea5abd-cc56-4d5c-977c-97403c1a78c4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.208965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:07.231427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.233542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.277074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.282899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.283195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.283477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.290704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.290704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.308099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 911/1828] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:07 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:07.310502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.326336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1580b9cc-ab13-4ec3-9916-c1b7ff7c8ac5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.332814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.334183 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.334461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.336258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] PATH_INFO: `/v3/projects/a8c25e2c7c7544d78b878797ee85ec01/users/032f54099115403f8c77d7bc5672de66/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.336979 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8c25e2c7c7544d78b878797ee85ec01, user_id=032f54099115403f8c77d7bc5672de66, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.340491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d2ea50-56e2-482a-8a29-b42d5e2848d1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.361269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 912/1829] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:07 2022] PUT /identity/v3/projects/a8c25e2c7c7544d78b878797ee85ec01/users/032f54099115403f8c77d7bc5672de66/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:07.370285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.370694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.374329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.409355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b98393-2418-4c3d-a5f1-e92195ff6835 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.433827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] MFA Rules not processed for user `032f54099115403f8c77d7bc5672de66`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:07.443338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.486361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.487300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.487687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.488185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] PATH_INFO: `/v3/projects/c2352ca767e244f9a63cd48d65e491b6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.489431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2352ca767e244f9a63cd48d65e491b6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.491512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.501654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:07.545148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 913/1830] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:07.556285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.557591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.614429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad2c6e-a29e-4426-adf8-266fc5b5b153 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.618144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.618787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.619070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.619371 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.629463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.631020 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56963bc9-0b5b-43d5-919e-c92d2d787cbc service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.641168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 914/1831] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:07.686547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2352ca767e244f9a63cd48d65e491b6', 'request_id': 'req-375afba9-fe51-4780-bc2c-88660027c853'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:07.764422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2352ca767e244f9a63cd48d65e491b6', 'request_id': 'req-375afba9-fe51-4780-bc2c-88660027c853'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:07.771968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2352ca767e244f9a63cd48d65e491b6', 'request_id': 'req-375afba9-fe51-4780-bc2c-88660027c853'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:07.783255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] The token cache is being invalidate because project c2352ca767e244f9a63cd48d65e491b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:07.783632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:07.784327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:07.785868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 918/1832] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:07 2022] DELETE /identity/v3/projects/c2352ca767e244f9a63cd48d65e491b6 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:07.791216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1ed5-7c29-457d-83c2-8b4e62be7312 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.791689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1ed5-7c29-457d-83c2-8b4e62be7312 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.791861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1ed5-7c29-457d-83c2-8b4e62be7312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.813790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bb1d1ed5-7c29-457d-83c2-8b4e62be7312 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:07.815076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bb1d1ed5-7c29-457d-83c2-8b4e62be7312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.841131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:07.854352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.855825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.880904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 915/1833] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:07.888680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.888964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.889247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.908744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:07.910016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.918136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 916/1834] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:07.922908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.927279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375afba9-fe51-4780-bc2c-88660027c853 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.931688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.931888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09759bc0-f094-4b94-992a-0b0513586460 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.932491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.932772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.938017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.938017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:07.938484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.939125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.939397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.939662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.940343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:07.940640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.942234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:07.958758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 917/1835] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:06:07.959586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:07.969260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae210d4-caba-469f-9fb1-19a966b70ffa admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:07.974490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:07.974859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:07.975147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:07.975423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:07.976073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:07.977994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.008344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.009616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.019794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 919/1836] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:08.089482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 918/1837] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:07 2022] POST /identity/v3/projects => generated 391 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:08.093007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:08.105006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346aad46-e0d0-49ae-886e-38bd76aa8415 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:08.170801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:08.177130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:08.177748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.178029 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.178304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.178971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.180967 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.268184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 920/1838] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:08 2022] POST /identity/v3/users => generated 296 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:08.272425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:08.283501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f70293a-8545-49d2-88b7-26875807da18 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:08.288435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:08.289119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.289470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.289807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.290736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.294240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.305813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 919/1839] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:08.307099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:08.318265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12062887-052e-4fb0-b6e7-93ad5f9bae0b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:08.324614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:08.326075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.326445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.326838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.327600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.329528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c43695-e7f5-40df-83fa-59dcb4d65e96 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.342183 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 921/1840] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:08.346170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:08.359049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbdfc3c-d852-4128-83de-52314a370c28 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:08.363745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:08.364395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.364695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.364969 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] PATH_INFO: `/v3/projects/aef317c3416342c8a6fd0acdf79bcf93/users/8ccf56a78148413cb41e208e9f892a57/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.381657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aef317c3416342c8a6fd0acdf79bcf93, user_id=8ccf56a78148413cb41e208e9f892a57, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.384497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.500161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 920/1841] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:08 2022] PUT /identity/v3/projects/aef317c3416342c8a6fd0acdf79bcf93/users/8ccf56a78148413cb41e208e9f892a57/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:08.513783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a09d-a234-4c5f-9918-04d9804f6362 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.514373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a09d-a234-4c5f-9918-04d9804f6362 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.514553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b225a09d-a234-4c5f-9918-04d9804f6362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.554716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b225a09d-a234-4c5f-9918-04d9804f6362 None None] MFA Rules not processed for user `8ccf56a78148413cb41e208e9f892a57`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:08.556085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b225a09d-a234-4c5f-9918-04d9804f6362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:08.638427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 922/1842] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:08.666661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:08.668955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:08.754830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7438aa-5dd7-4e01-aca0-8efe5001e271 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:08.758534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:08.759179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:08.759460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:08.759731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:08.768294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:08.769731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:08.781363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 921/1843] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:14.900395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fe1a-59ff-48f4-b3ca-47ea67e9edd3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:14.900947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fe1a-59ff-48f4-b3ca-47ea67e9edd3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:14.900947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b519fe1a-59ff-48f4-b3ca-47ea67e9edd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:14.923832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-b519fe1a-59ff-48f4-b3ca-47ea67e9edd3 None None] MFA Rules not processed for user `6b59e9b0ec6e4fee99a93c58f9dcd6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:14.925502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b519fe1a-59ff-48f4-b3ca-47ea67e9edd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:14.992545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 923/1844] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:15.493423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:15.494991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:15.564460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:15.570349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ad3bf0-03e7-4521-9f10-5e87455c4eee service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:15.575199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:15.575888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:15.576164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:15.576440 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] PATH_INFO: `/v3/users/10d81d81253b4c0d98088ce2e96ee337` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:15.577622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10d81d81253b4c0d98088ce2e96ee337)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:15.579715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:15.831012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10d81d81253b4c0d98088ce2e96ee337'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:15.831934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10d81d81253b4c0d98088ce2e96ee337'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:15.848415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10d81d81253b4c0d98088ce2e96ee337'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:15.924174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10d81d81253b4c0d98088ce2e96ee337'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:15.924922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10d81d81253b4c0d98088ce2e96ee337'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:15.947525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 922/1845] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:15 2022] DELETE /identity/v3/users/10d81d81253b4c0d98088ce2e96ee337 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:15.952433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:15.952906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:15.952998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:15.989807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:15.991216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:16.062043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 924/1846] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:16.068760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:16.070182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:16.151221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e72b0-cdfc-420b-a7b6-b6ecdf5c7d99 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:16.162541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:16.163205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:16.163567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:16.163964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:16.183607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:16.183607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:16.191292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 923/1847] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:16.675312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:16.678486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:16.685796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:16.690471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee576bb1-cae6-4e8f-8df5-3925e65637d2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:16.697089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:16.697725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:16.698033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:16.698278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] PATH_INFO: `/v3/projects/e60eb360cc404961a14cb18e1e2a4c6d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:16.699463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60eb360cc404961a14cb18e1e2a4c6d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:16.702759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:16.844740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60eb360cc404961a14cb18e1e2a4c6d', 'request_id': 'req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:16.906605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60eb360cc404961a14cb18e1e2a4c6d', 'request_id': 'req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:16.914284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60eb360cc404961a14cb18e1e2a4c6d', 'request_id': 'req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:16.926821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] The token cache is being invalidate because project e60eb360cc404961a14cb18e1e2a4c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:16.927115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:16.927920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:16.929425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 925/1848] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:16 2022] DELETE /identity/v3/projects/e60eb360cc404961a14cb18e1e2a4c6d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:16.933767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:16.946685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93638b4b-c565-48d2-a637-2ccfa045186a service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:17.020458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:17.021130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:17.021539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:17.021823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] PATH_INFO: `/v3/users/032f54099115403f8c77d7bc5672de66` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:17.022988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=032f54099115403f8c77d7bc5672de66)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:17.024953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:17.052527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:17.260004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '032f54099115403f8c77d7bc5672de66'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:17.260686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '032f54099115403f8c77d7bc5672de66'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:17.274033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '032f54099115403f8c77d7bc5672de66'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:17.341289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '032f54099115403f8c77d7bc5672de66'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:17.341932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0588a721-650d-47ac-b3df-20b0431adc23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '032f54099115403f8c77d7bc5672de66'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:17.358042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 924/1849] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:16 2022] DELETE /identity/v3/users/032f54099115403f8c77d7bc5672de66 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:17.781953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:17.783472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:17.797674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7d34ff-0fea-49bc-b15d-fbe7304bade5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:17.854546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:17.855211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:17.855491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:17.855764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] PATH_INFO: `/v3/projects/a8c25e2c7c7544d78b878797ee85ec01` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:17.857471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8c25e2c7c7544d78b878797ee85ec01)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:17.859416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:17.987075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8c25e2c7c7544d78b878797ee85ec01', 'request_id': 'req-06c294ca-084f-4817-a752-c65ec9dda839'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:18.066215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8c25e2c7c7544d78b878797ee85ec01', 'request_id': 'req-06c294ca-084f-4817-a752-c65ec9dda839'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:18.072819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8c25e2c7c7544d78b878797ee85ec01', 'request_id': 'req-06c294ca-084f-4817-a752-c65ec9dda839'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:18.083503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] The token cache is being invalidate because project a8c25e2c7c7544d78b878797ee85ec01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:18.083796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:18.084424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-06c294ca-084f-4817-a752-c65ec9dda839 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:18.085905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 926/1850] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:17 2022] DELETE /identity/v3/projects/a8c25e2c7c7544d78b878797ee85ec01 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:18.091531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.091984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.092076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.112920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:18.114220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:18.175200 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 925/1851] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:18.183002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.183288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.183565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.204006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:18.205202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:18.213176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 927/1852] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:18.217639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.227171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4212cb-ae11-4305-996f-c00641789e35 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.232656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.233260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.233541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.233813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.234460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:18.234803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.236403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.255846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 926/1853] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:18.257045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.267614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adefd56-53f8-4a4f-a494-5d91ff3dcf8c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.272281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.272883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.273157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.273429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.274053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.276112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.287355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:18.384673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 928/1854] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:18.388743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.399158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca1adf5-3452-43e8-87cd-105941aa2a3f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.470508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.471547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.472122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.472575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.473642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.475697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.564190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 927/1855] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:18 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:18.568386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.578689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65d1ca-72c4-4d7a-bc8b-d673d686ee8a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.583757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.585424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.585711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.585989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.586669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.588268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.598645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 929/1856] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:18.602546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.612909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac79c24-2597-4e63-aa44-0a9fce76eaba admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.617629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.618216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.618493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.618797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.619437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.621008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824379d-9e64-41b8-9d60-8f5a9374a492 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.632331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 928/1857] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:18.636526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.647330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46297ea5-af2c-43cc-a2a7-3981ca4f8ed7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.651941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.652575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.652857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.653588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] PATH_INFO: `/v3/projects/55dcd53088bc428d86d11d4129853f63/users/96cbbecec36b42baa0e82ed09095528f/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.668217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55dcd53088bc428d86d11d4129853f63, user_id=96cbbecec36b42baa0e82ed09095528f, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.670937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.758039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 930/1858] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:18 2022] PUT /identity/v3/projects/55dcd53088bc428d86d11d4129853f63/users/96cbbecec36b42baa0e82ed09095528f/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:18.770788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.771074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.771357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.779145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:18.800320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] MFA Rules not processed for user `96cbbecec36b42baa0e82ed09095528f`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:18.801540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:18.869550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 929/1859] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:18.881528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:18.883271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:18.941269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44333b6-5dc9-459e-8f13-b794da74e047 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:18.945364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:18.945967 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:18.946380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:18.946693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:18.955361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:18.956856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:18.965671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 931/1860] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:28.936129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=31 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:28.961664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=45,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:31.344747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:31.467996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99f413-d5cc-43dc-9586-b626f5989902 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:31.472848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:31.473678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:31.474023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:31.474387 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] PATH_INFO: `/v3/users/96cbbecec36b42baa0e82ed09095528f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:31.475560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] RBAC: Authorizing `identity:delete_user(user_id=96cbbecec36b42baa0e82ed09095528f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:31.477583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:31.753933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96cbbecec36b42baa0e82ed09095528f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:31.754590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96cbbecec36b42baa0e82ed09095528f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:31.767591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96cbbecec36b42baa0e82ed09095528f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:31.802478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:31.804094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:31.818296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8498f7f4-cbf8-4e97-8e23-66fc8f2968b5 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:31.822865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:31.823491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:31.823769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:31.824041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] PATH_INFO: `/v3/users/8ccf56a78148413cb41e208e9f892a57` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:31.825741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ccf56a78148413cb41e208e9f892a57)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:31.827680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:31.866938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96cbbecec36b42baa0e82ed09095528f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:31.867593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f83b89d8-22e6-4f6c-a4fe-cc70f506face admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96cbbecec36b42baa0e82ed09095528f'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:31.880619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 930/1861] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:31 2022] DELETE /identity/v3/users/96cbbecec36b42baa0e82ed09095528f => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:31.885168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:31.885456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:31.885761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:31.913151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:31.914376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:31.978270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 931/1862] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:31.993047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:31.994411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:32.022501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:32.066467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501bd49b-9b6f-4d34-837b-49ace6522991 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:32.070373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:32.070530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ccf56a78148413cb41e208e9f892a57'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:32.071266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:32.071836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:32.072752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:32.087758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ccf56a78148413cb41e208e9f892a57'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:32.088045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:32.088510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ccf56a78148413cb41e208e9f892a57'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:32.092406 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a641155-eff1-4519-9a09-693a50df316c service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:32.096486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ccf56a78148413cb41e208e9f892a57'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:32.097194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ccf56a78148413cb41e208e9f892a57'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:32.105796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 932/1863] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:32.206879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 932/1864] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:31 2022] DELETE /identity/v3/users/8ccf56a78148413cb41e208e9f892a57 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:32.211569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:32.212021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:32.212328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:32.244100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:32.246013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:32.351747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 933/1865] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:32.376332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:32.378668 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:32.482030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd106b77-d49e-4391-ba02-87e555b390d0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:32.488963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:32.490249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:32.490786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:32.491361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:32.507345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:32.508892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73fe9-ba4c-4e9c-959d-5e52f8d31cef service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:32.523496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 933/1866] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:32.703023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:32.704870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:32.724732 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a128-0df2-499e-94bc-de499a1c01f9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:32.744972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:32.745740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:32.746019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:32.746288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] PATH_INFO: `/v3/projects/55dcd53088bc428d86d11d4129853f63` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:32.748786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55dcd53088bc428d86d11d4129853f63)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:32.751710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:32.895657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55dcd53088bc428d86d11d4129853f63', 'request_id': 'req-9106dbda-3d93-4081-8392-5e6fdb6a5078'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.021030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55dcd53088bc428d86d11d4129853f63', 'request_id': 'req-9106dbda-3d93-4081-8392-5e6fdb6a5078'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.027292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55dcd53088bc428d86d11d4129853f63', 'request_id': 'req-9106dbda-3d93-4081-8392-5e6fdb6a5078'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.040640 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] The token cache is being invalidate because project 55dcd53088bc428d86d11d4129853f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:33.040931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.041587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-9106dbda-3d93-4081-8392-5e6fdb6a5078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.043209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 934/1867] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:32 2022] DELETE /identity/v3/projects/55dcd53088bc428d86d11d4129853f63 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:33.049649 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.050178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.050393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.074311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:33.076287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:33.144588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 934/1868] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:33.152326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.162062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.162062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.174522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:33.175849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:33.184547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 935/1869] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:33.193979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.204713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddc222-0c88-4b4a-8998-b0ef20463fe2 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.209597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.210218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.210501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.210806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.211456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:33.211760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.213368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.225763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 935/1870] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:33.229995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.249912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43404e6b-8b2c-4f8c-acc0-a55a9021d177 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.255454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.256080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.256359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.256630 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.257340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.259330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.334531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 936/1871] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/projects => generated 375 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:33.338566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.344918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:33.350092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76acf4c8-5321-4463-8ce6-2271b3c24bfa admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.417383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.418099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.418424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.422883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.422883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.422883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.444637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.446614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:33.461906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1070-439c-497a-b4a4-2423cdcf175b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.469222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.469941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.470333 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.470613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] PATH_INFO: `/v3/projects/aef317c3416342c8a6fd0acdf79bcf93` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.472080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aef317c3416342c8a6fd0acdf79bcf93)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.473995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.488531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:33.549568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 936/1872] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/users => generated 288 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:33.553654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.570335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646023eb-b397-4133-bb7c-3a9cb5c55f31 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.572934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.573658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.574028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.574405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.576162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.577935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.591481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 937/1873] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:33.594839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.605247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306450f5-55f8-4ad4-a395-9a1ff0dae0f1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.612366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aef317c3416342c8a6fd0acdf79bcf93', 'request_id': 'req-beeea13d-6185-4059-b2ae-9de3f958a8a7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.666805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aef317c3416342c8a6fd0acdf79bcf93', 'request_id': 'req-beeea13d-6185-4059-b2ae-9de3f958a8a7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.667022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.667525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.667813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.668087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.668728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.670579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.673635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aef317c3416342c8a6fd0acdf79bcf93', 'request_id': 'req-beeea13d-6185-4059-b2ae-9de3f958a8a7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.680645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 938/1874] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:33.685465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.686655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] The token cache is being invalidate because project aef317c3416342c8a6fd0acdf79bcf93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:33.687046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.687864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-beeea13d-6185-4059-b2ae-9de3f958a8a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:33.689613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 937/1875] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:33 2022] DELETE /identity/v3/projects/aef317c3416342c8a6fd0acdf79bcf93 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:33.695722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc937f-4bb8-44d4-a34b-111f1ec1ae4d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:33.702325 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:33.703069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.703350 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.703696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] PATH_INFO: `/v3/projects/f872dfddccaa4b2a8f75216ba7e2b5bf/users/7313dfc94cf042d2b2a135934d575335/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.726555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f872dfddccaa4b2a8f75216ba7e2b5bf, user_id=7313dfc94cf042d2b2a135934d575335, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:33.730202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:33.823496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 939/1876] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:33 2022] PUT /identity/v3/projects/f872dfddccaa4b2a8f75216ba7e2b5bf/users/7313dfc94cf042d2b2a135934d575335/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:33.835441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:33.835735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:33.836192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:33.871637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] MFA Rules not processed for user `7313dfc94cf042d2b2a135934d575335`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:33.873681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:33.941207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 938/1877] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:33.945925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:33.947621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:34.024736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745577ec-8584-406d-86fe-0c8d30a85d1f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.030991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.031632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.031912 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.032196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.033379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.035893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1da939-2eb6-4b3c-9a62-c238be919fae admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.041034 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:34.188209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 940/1878] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:33 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:34.189415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.204642 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f67213-ee3d-4173-9a08-b0c0cdcb96f5 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.206207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.206503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.206832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.227329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] MFA Rules not processed for user `1eb3641974744d83816b6f3ad7bb6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:34.228570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:34.274812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.275574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.276002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.276376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.277208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.280166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.302598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 941/1879] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:34.433240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 939/1880] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:34 2022] POST /identity/v3/users => generated 294 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:34.456922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.475248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63425fc3-779f-4be7-a555-e04494c2d4dd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.480803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.482113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.482472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.482910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.483868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.485771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.499665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 942/1881] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:34.503556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.514469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72643768-c9f5-4d4f-9f15-12d4e4b0920f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.519442 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.520119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.520396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.520887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] PATH_INFO: `/v3/projects/88413dedbc4c43cebe061b7ca58aed52/users/ef4c86b1c8cc4d2f903663102483d81c/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.539156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88413dedbc4c43cebe061b7ca58aed52, user_id=ef4c86b1c8cc4d2f903663102483d81c, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.542431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.546814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:34.623174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 940/1882] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:34 2022] PUT /identity/v3/projects/88413dedbc4c43cebe061b7ca58aed52/users/ef4c86b1c8cc4d2f903663102483d81c/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:34.627465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.639157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2844a63-b663-481e-8609-f1bf71667799 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.699697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.700911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.701363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.701584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.702283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.703985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606aecf-8462-4bda-93d9-5f488d151ac9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.722420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 943/1883] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:34.726728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.737463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d30e71-049b-479b-8783-67833f6fb804 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:34.742113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:34.742810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.743094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.743392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] PATH_INFO: `/v3/projects/88413dedbc4c43cebe061b7ca58aed52/users/ef4c86b1c8cc4d2f903663102483d81c/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.745365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88413dedbc4c43cebe061b7ca58aed52, user_id=ef4c86b1c8cc4d2f903663102483d81c, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:34.748272 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:34.759247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 941/1884] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:34 2022] PUT /identity/v3/projects/88413dedbc4c43cebe061b7ca58aed52/users/ef4c86b1c8cc4d2f903663102483d81c/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:34.833496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:34.833780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:34.834063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:34.865906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] MFA Rules not processed for user `ef4c86b1c8cc4d2f903663102483d81c`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:34.867162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:34.939399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:34.950908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 944/1885] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:34 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:34.961490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:34.963227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:35.039829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff13cd-ebea-4470-9fc9-3ac93cc086d4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:35.045922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:35.046564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:35.046886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:35.047167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:35.119401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:35.120890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:35.131980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 942/1886] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:38.464205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:38.537306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f78c45-6101-4738-b854-3e2e9d63bc42 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:38.542049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:38.542726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:38.543001 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:38.543274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] PATH_INFO: `/v3/users/7313dfc94cf042d2b2a135934d575335` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:38.544443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7313dfc94cf042d2b2a135934d575335)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:38.546509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:38.619989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:38.621518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:38.637173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3aa5c-dae1-4d9a-89ef-2cefeb2c7d96 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:38.643656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:38.643656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:38.643974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:38.644158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] PATH_INFO: `/v3/users/9b160e28dc6244c7aaf53d1fa165efc2` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:38.645342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b160e28dc6244c7aaf53d1fa165efc2)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:38.647335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:38.813863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7313dfc94cf042d2b2a135934d575335'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.828753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7313dfc94cf042d2b2a135934d575335'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.832776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7313dfc94cf042d2b2a135934d575335'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.837432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7313dfc94cf042d2b2a135934d575335'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.838166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-07727e45-7db0-44ed-baa3-39389d14008c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7313dfc94cf042d2b2a135934d575335'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.907078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b160e28dc6244c7aaf53d1fa165efc2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.907760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b160e28dc6244c7aaf53d1fa165efc2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.923167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b160e28dc6244c7aaf53d1fa165efc2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:38.923991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 945/1887] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:38 2022] DELETE /identity/v3/users/7313dfc94cf042d2b2a135934d575335 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:38.925544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:38.925691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:38.925968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:38.952511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:38.954548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.009722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b160e28dc6244c7aaf53d1fa165efc2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:39.010518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-aac5ecfd-c956-43ac-894a-0dc599ec27ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b160e28dc6244c7aaf53d1fa165efc2'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:39.027830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 943/1888] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:38 2022] DELETE /identity/v3/users/9b160e28dc6244c7aaf53d1fa165efc2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:39.032991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:39.033547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:39.034162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:39.060071 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:39.063422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.064910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 946/1889] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:39.068526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:39.078985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:39.081402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.121251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:39.208215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0579914f-b858-4200-8950-fb652bd65d49 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:39.224355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 944/1890] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:39.240002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:39.241864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.291163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:39.292947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:39.293484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:39.295095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:39.310888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:39.312678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa09193-3655-40b3-a9c5-edd018d12464 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:39.324020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 947/1891] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:39.334062 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adceaa-d527-4650-8540-191342c9e5c9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:39.339563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:39.340298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:39.340647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:39.341198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:39.352687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:39.354270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:39.365850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 945/1892] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:39.877323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:39.877323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:39.877323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:39.897807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] MFA Rules not processed for user `8632f75445ef4fd3a2f1ebbc2ed87e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:39.899113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.967192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:39.968783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:39.983965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbef73-d9be-4ce0-96d1-7725175a1701 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:39.990683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 948/1893] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:39.990852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:39.990852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:39.990852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:39.990852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] PATH_INFO: `/v3/projects/f872dfddccaa4b2a8f75216ba7e2b5bf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:39.993731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f872dfddccaa4b2a8f75216ba7e2b5bf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:39.995697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:40.051992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:40.054497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:40.077439 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999a77e-4a70-477f-8652-f2ecb6a3d047 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:40.090406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:40.091763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.094750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.094750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] PATH_INFO: `/v3/projects/b3f8918528774d3c89b883f4cc7eb0b5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.094750 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3f8918528774d3c89b883f4cc7eb0b5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:40.095983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:40.157724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f872dfddccaa4b2a8f75216ba7e2b5bf', 'request_id': 'req-214da3a3-5a70-427c-aa13-79f437b0dc59'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.284745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f872dfddccaa4b2a8f75216ba7e2b5bf', 'request_id': 'req-214da3a3-5a70-427c-aa13-79f437b0dc59'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.297588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f872dfddccaa4b2a8f75216ba7e2b5bf', 'request_id': 'req-214da3a3-5a70-427c-aa13-79f437b0dc59'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.311258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] The token cache is being invalidate because project f872dfddccaa4b2a8f75216ba7e2b5bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:40.311547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.312450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.314388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 946/1894] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:39 2022] DELETE /identity/v3/projects/f872dfddccaa4b2a8f75216ba7e2b5bf => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:40.317608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3f8918528774d3c89b883f4cc7eb0b5', 'request_id': 'req-73069b28-840f-4213-8978-447566bc9894'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.318688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:40.320382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:40.375405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:40.405821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214da3a3-5a70-427c-aa13-79f437b0dc59 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:40.444294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3f8918528774d3c89b883f4cc7eb0b5', 'request_id': 'req-73069b28-840f-4213-8978-447566bc9894'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.454017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3f8918528774d3c89b883f4cc7eb0b5', 'request_id': 'req-73069b28-840f-4213-8978-447566bc9894'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.467740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:40.475570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] The token cache is being invalidate because project b3f8918528774d3c89b883f4cc7eb0b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:40.477621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.478986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73069b28-840f-4213-8978-447566bc9894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.481528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 949/1895] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:40 2022] DELETE /identity/v3/projects/b3f8918528774d3c89b883f4cc7eb0b5 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:40.487518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0e40d-a683-43e2-836a-7eb45bc5d8d1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.487815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0e40d-a683-43e2-836a-7eb45bc5d8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.488125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b0e40d-a683-43e2-836a-7eb45bc5d8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.492785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:40.493572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.493952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.494328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] PATH_INFO: `/v3/users/ef4c86b1c8cc4d2f903663102483d81c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.495829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef4c86b1c8cc4d2f903663102483d81c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:40.499552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:40.520301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-85b0e40d-a683-43e2-836a-7eb45bc5d8d1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:40.521768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-85b0e40d-a683-43e2-836a-7eb45bc5d8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:40.533186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 950/1896] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:40.544905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.545217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.545514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.566835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:40.568219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:40.577231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 951/1897] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:40.583151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:40.593524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e30291-e1a7-457c-ab6d-1dd25e102aac None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:40.598360 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:40.599035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.599948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.600236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.600892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:40.601229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:40.602877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:40.619865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 952/1898] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:40.621140 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:40.635910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40a820-f559-477b-a9b5-3f41126b9e0f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:40.638316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:40.639429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.639429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.639647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.640633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:40.642449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:40.711588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef4c86b1c8cc4d2f903663102483d81c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.712217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef4c86b1c8cc4d2f903663102483d81c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.730558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef4c86b1c8cc4d2f903663102483d81c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.788004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 953/1899] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:40 2022] POST /identity/v3/projects => generated 399 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:40.789206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:40.804283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef4c86b1c8cc4d2f903663102483d81c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.805354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef4c86b1c8cc4d2f903663102483d81c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:40.818957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 947/1900] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:40 2022] DELETE /identity/v3/users/ef4c86b1c8cc4d2f903663102483d81c => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:40.860536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c0fb85-6413-4ead-b740-e28880d8d192 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:40.934275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:40.935044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:40.935123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:40.935365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:40.936506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:40.939566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.085532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 954/1901] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:40 2022] POST /identity/v3/users => generated 300 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:41.089873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:41.092310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:41.108776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b656c3-7ceb-4d6f-a77e-5815f062fe61 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:41.113490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:41.114115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.114402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.114704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.115353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:41.116974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.130671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 948/1902] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:41.135023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:41.151934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726a9529-98e3-4604-b41e-91ea4e0b4f52 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:41.157467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:41.158141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.158498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.159084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.159404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:41.161054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b94051b-cb7c-4438-b06b-9042746f4c86 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.176992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 955/1903] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:41.181286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:41.194938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a4194-ef51-4423-90e9-6bc72eb4363d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:41.201089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:41.201545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.201819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.202094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] PATH_INFO: `/v3/projects/20509f4993cb4e27bec33a73f903d7e2/users/88ec2d66abd946ca8526625471e9ee7e/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.221361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20509f4993cb4e27bec33a73f903d7e2, user_id=88ec2d66abd946ca8526625471e9ee7e, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:41.224207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.324013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 949/1904] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:41 2022] PUT /identity/v3/projects/20509f4993cb4e27bec33a73f903d7e2/users/88ec2d66abd946ca8526625471e9ee7e/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:41.335083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.335372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.335648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.366299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] MFA Rules not processed for user `88ec2d66abd946ca8526625471e9ee7e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:41.367587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:41.479115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 956/1905] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:41 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:41.486955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:41.495381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:41.506583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:41.508262 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:41.575280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def92bd1-9195-495c-a0dc-1acba4824330 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:41.586467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:41.586467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.586467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.586467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.591230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:41.592954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22c3bd9-4a17-4883-a158-69dab37944a9 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.617525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 950/1906] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:41.619617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:41.635182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ca279-0ec4-4612-8613-287a488af69f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:41.642802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:41.643472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:41.643763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:41.644037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] PATH_INFO: `/v3/projects/88413dedbc4c43cebe061b7ca58aed52` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:41.645326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88413dedbc4c43cebe061b7ca58aed52)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:41.647190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:41.838976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88413dedbc4c43cebe061b7ca58aed52', 'request_id': 'req-f84bb203-3b09-4b5f-8f81-21b54febbb98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:41.878537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:41.999143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88413dedbc4c43cebe061b7ca58aed52', 'request_id': 'req-f84bb203-3b09-4b5f-8f81-21b54febbb98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:42.006761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88413dedbc4c43cebe061b7ca58aed52', 'request_id': 'req-f84bb203-3b09-4b5f-8f81-21b54febbb98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:42.020846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] The token cache is being invalidate because project 88413dedbc4c43cebe061b7ca58aed52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:42.020846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:42.021267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f84bb203-3b09-4b5f-8f81-21b54febbb98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:42.026334 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 957/1907] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:41 2022] DELETE /identity/v3/projects/88413dedbc4c43cebe061b7ca58aed52 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:06:42.028251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.028609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.028963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.054763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:42.056147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:42.129598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 951/1908] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:06:42.137382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.138322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.138607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.159580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:42.160675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:42.171742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 958/1909] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:42.176281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.193000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c6a8b-a232-4d92-8f1e-ef2d1c810836 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.199056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.199911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.200244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.200520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.201863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:42.202195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.204675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.218479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 952/1910] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:42.222852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.232930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d68bb5-d475-47b0-819c-6edf75aed189 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.237586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.238197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.238471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.238774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.239597 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.241926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.362070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 959/1911] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:42 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:42.365902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.377117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77298ec-e49a-4f82-88a3-cdb3a3e21c67 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.443128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.443779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.444028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.444300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.444935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.446903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.541424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 953/1912] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:42 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:42.542737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.563017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f1fc3-ca48-40cd-9c54-7636860c0cde admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.568853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.569579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.569941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.570205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.575273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.575273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.596841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 960/1913] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:42.596945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.599871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c105a94-bbc8-4f47-868e-99ae7199aebb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.604709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.605411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.605700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.605976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.607246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.608875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b8a8e-2380-41cb-9f98-bb2cab646c58 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.620270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 954/1914] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:42.624266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.634622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6287025d-c4da-4fca-8f4a-a8ba809ef840 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.639650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.640277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.640557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.640830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] PATH_INFO: `/v3/projects/57eb0443c72743c8b3ab016fdf5c9745/users/f71ffc3d05d4470fa533cfd6f22184b2/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.656746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57eb0443c72743c8b3ab016fdf5c9745, user_id=f71ffc3d05d4470fa533cfd6f22184b2, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.658843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.764105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 961/1915] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:42 2022] PUT /identity/v3/projects/57eb0443c72743c8b3ab016fdf5c9745/users/f71ffc3d05d4470fa533cfd6f22184b2/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:42.769347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f667ff-2de0-4aca-8915-eff6090b201a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.769627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f667ff-2de0-4aca-8915-eff6090b201a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.769896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f667ff-2de0-4aca-8915-eff6090b201a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.789936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-93f667ff-2de0-4aca-8915-eff6090b201a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:42.791423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-93f667ff-2de0-4aca-8915-eff6090b201a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:42.850089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 955/1916] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:42.860968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:42.862868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:42.920177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5232eb-b8c4-405d-803a-59c099ba9ca5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:42.923959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:42.924568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:42.924846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:42.925119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:42.934964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:42.936040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:42.946232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 962/1917] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:42.975330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:43.426292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:48.611259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a628b-fd35-432b-877b-ca022a03d80a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:48.612315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a628b-fd35-432b-877b-ca022a03d80a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:48.612774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a628b-fd35-432b-877b-ca022a03d80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:48.651134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2f9a628b-fd35-432b-877b-ca022a03d80a None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:48.656257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2f9a628b-fd35-432b-877b-ca022a03d80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:48.742684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 956/1918] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:06:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:48.834103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:48.837797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:48.959278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3040b3-156c-49ae-85ea-950c3f8f6309 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:48.968897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:48.969916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:48.970481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:48.971674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:48.987115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:48.987385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:49.060770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 963/1919] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:06:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:06:50.258537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:50.259505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:50.259997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:50.335120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] MFA Rules not processed for user `f71ffc3d05d4470fa533cfd6f22184b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:50.336744 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:50.467537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 957/1920] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:50.468514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:50.482495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de612bf-9415-411a-9649-fff437e93bc0 service swift] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:50.488693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:50.489955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:50.490242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:50.490515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:50.491219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:50.495633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:51.220727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 964/1921] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:50 2022] POST /identity/v3/projects => generated 377 bytes in 753 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:51.225065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:51.243317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d03fc-770a-45e3-a6d0-03ebe7590384 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:51.360890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:51.362839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:51.365721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:51.367470 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:51.369487 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:51.369976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:51.373464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:51.377370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:51.384961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba816011-e467-40c7-a059-0e95af194724 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:51.395369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:51.397096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:51.399096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:51.399382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] PATH_INFO: `/v3/users/88ec2d66abd946ca8526625471e9ee7e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:51.401425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ec2d66abd946ca8526625471e9ee7e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:51.408426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:51.504620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:51.518388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 958/1922] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:51 2022] POST /identity/v3/users => generated 295 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:51.520948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:51.542053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd319136-e380-4e63-9743-79c0c8e11654 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:51.549544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:51.550748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:51.556420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:51.556420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:51.556420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:51.556420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:51.585910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 959/1923] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:51.587567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:51.617005 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd540b-db9a-4c20-b4bc-e174b6903b3c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:51.629549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:51.630281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:51.630559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:51.630879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] PATH_INFO: `/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766/users/6e3aafdb629044f78c2b2e289e5409d3/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:51.663167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb6b4f5f9aa4202a7104f39187d5766, user_id=6e3aafdb629044f78c2b2e289e5409d3, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:51.668441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:51.838899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 960/1924] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:51 2022] PUT /identity/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766/users/6e3aafdb629044f78c2b2e289e5409d3/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:51.842971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:51.865732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ec2d66abd946ca8526625471e9ee7e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:51.905493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:51.914114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ec2d66abd946ca8526625471e9ee7e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:51.916519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ec2d66abd946ca8526625471e9ee7e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:51.946064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ec2d66abd946ca8526625471e9ee7e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:51.948218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7785c85d-1f46-4db3-8b9f-59ee0706cf97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ec2d66abd946ca8526625471e9ee7e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:51.987199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0b7dc-e609-45b4-b76f-6e739f364153 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:52.000996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:52.001638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:52.001915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:52.002189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:52.002923 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:52.005460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:52.026131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 961/1925] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:52.038401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:52.045147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:52.070246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 965/1926] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:06:51 2022] DELETE /identity/v3/users/88ec2d66abd946ca8526625471e9ee7e => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:52.075931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3d6aa-cf80-48a9-87bb-a4ac2353842f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:52.077859 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:52.081133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:52.081698 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:52.171730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:52.171730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:52.209924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:52.210886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:52.211361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:52.212145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] PATH_INFO: `/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766/users/6e3aafdb629044f78c2b2e289e5409d3/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:52.222990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cb6b4f5f9aa4202a7104f39187d5766, user_id=6e3aafdb629044f78c2b2e289e5409d3, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:52.225934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:52.258797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 962/1927] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:52 2022] PUT /identity/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766/users/6e3aafdb629044f78c2b2e289e5409d3/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:52.310872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 966/1928] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:52.350649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:52.353155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:52.540389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d73cdc4-4897-45d4-a3d4-747f226cb1a1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:52.551755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:52.552407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:52.552684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:52.552961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:52.571279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:52.716708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:52.718257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ec3ec-f062-4ef2-94d4-c670e3f571f2 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:52.745783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 963/1929] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:06:54.750107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:54.752794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:54.777598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5793b63-25a0-4a22-8c64-7df4d0f66a6a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:54.787870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:54.789030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:54.789574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:54.790131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] PATH_INFO: `/v3/projects/20509f4993cb4e27bec33a73f903d7e2` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:54.792060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] RBAC: Authorizing `identity:delete_project(project_id=20509f4993cb4e27bec33a73f903d7e2)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:54.794560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:55.018860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20509f4993cb4e27bec33a73f903d7e2', 'request_id': 'req-710cc980-b0e8-4e2c-b5df-c9801394948b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:55.187414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20509f4993cb4e27bec33a73f903d7e2', 'request_id': 'req-710cc980-b0e8-4e2c-b5df-c9801394948b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:55.197176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20509f4993cb4e27bec33a73f903d7e2', 'request_id': 'req-710cc980-b0e8-4e2c-b5df-c9801394948b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:55.225181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] The token cache is being invalidate because project 20509f4993cb4e27bec33a73f903d7e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:06:55.225520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:55.227096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-710cc980-b0e8-4e2c-b5df-c9801394948b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:06:55.233486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 967/1930] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:06:54 2022] DELETE /identity/v3/projects/20509f4993cb4e27bec33a73f903d7e2 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:55.236737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:55.237377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:55.237821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:55.285746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:55.287548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:55.499323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 964/1931] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:55.528640 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:55.529088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:55.529288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:55.556060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:55.557332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:55.570301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 968/1932] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:55.571562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:55.595673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facba0dc-f427-4a12-97e0-88f4994ef04d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:55.619514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:55.620345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:55.620766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:55.621185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:55.622716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:06:55.623535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:55.626779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:55.648751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 965/1933] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:06:55.649671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:55.660744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce91ba99-1dc3-4481-8234-d78bd2ac1cf7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:55.669504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:55.670602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:55.671078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:55.671478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:55.672500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:55.675274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:56.096246 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 969/1934] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:06:55 2022] POST /identity/v3/projects => generated 383 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:06:56.100030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:56.110753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6327e48d-3411-426b-8fcc-28e910d5b8a6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:56.251791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:56.253319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:56.254663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:56.255741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:56.260239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:56.260239 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:56.353217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 966/1935] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:06:56 2022] POST /identity/v3/users => generated 292 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:06:56.357872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:56.381617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d0ed2-6855-4d21-b1ee-6386b71d21c5 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:56.396049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:56.396710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:56.397053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:56.397336 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:56.398003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:56.399658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:56.419880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 970/1936] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:06:56.423940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:56.444299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc1410-8e51-4a1e-81d3-44e5d01b72d1 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:56.457806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:56.458473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:56.459824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:56.459824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:56.459824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:56.461780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9869e7-a561-40ca-9572-a50dcbf33c11 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:56.493596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 967/1937] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:06:56.497988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:56.513486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b369e5-c15e-4835-b715-5e041b6a1465 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:56.523481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:56.526972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:56.527292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:56.527581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] PATH_INFO: `/v3/projects/a77531edf7224da8847bb5c9813ae7da/users/2f95a367dd3148f1b31363513af8b637/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:56.558374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a77531edf7224da8847bb5c9813ae7da, user_id=2f95a367dd3148f1b31363513af8b637, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:56.567170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:56.572388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:56.738442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 971/1938] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:06:56 2022] PUT /identity/v3/projects/a77531edf7224da8847bb5c9813ae7da/users/2f95a367dd3148f1b31363513af8b637/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:06:56.740220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d182ca-b555-4ed1-8ba9-2c89ef77b59e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:56.740534 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d182ca-b555-4ed1-8ba9-2c89ef77b59e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:56.740813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d182ca-b555-4ed1-8ba9-2c89ef77b59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:56.793172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f9d182ca-b555-4ed1-8ba9-2c89ef77b59e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:56.795579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f9d182ca-b555-4ed1-8ba9-2c89ef77b59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:56.816736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:06:56.975720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 968/1939] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:56.988365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:56.990209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:57.190258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaee988-1ad9-422a-b836-3109672227b7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:57.197589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:57.198566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:57.199020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:57.199608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:57.223374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:57.225443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:57.278226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 972/1940] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:06:59.041398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39e27-2140-4d2d-91e9-a7b586373aed None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:59.041919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39e27-2140-4d2d-91e9-a7b586373aed None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:59.041919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff39e27-2140-4d2d-91e9-a7b586373aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:59.069230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dff39e27-2140-4d2d-91e9-a7b586373aed None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:06:59.071266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dff39e27-2140-4d2d-91e9-a7b586373aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:59.172604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 969/1941] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:06:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:06:59.223421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:06:59.225609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:06:59.391669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398b8ad-6508-4a7c-a275-6636cf639e16 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:06:59.398932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:06:59.399718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:06:59.400090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:06:59.400486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:06:59.411981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:06:59.414310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:06:59.441746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 973/1942] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:07:05.160698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc63eba-53bc-4887-9792-d90cb131b97d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:05.161276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc63eba-53bc-4887-9792-d90cb131b97d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:05.161614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc63eba-53bc-4887-9792-d90cb131b97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:05.249614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dbc63eba-53bc-4887-9792-d90cb131b97d None None] MFA Rules not processed for user `6e3aafdb629044f78c2b2e289e5409d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:05.251105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dbc63eba-53bc-4887-9792-d90cb131b97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:05.487189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 970/1943] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:06.245102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:06.245968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:06.413636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada01750-fcfe-4ef1-a490-07ebc6f145e9 service swift] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:06.432780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:06.439118 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:06.439589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:06.439968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:06.443279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:06.544606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:06.657058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:06.658785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:06.697278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 974/1944] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:13.709328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:13.710307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:13.710868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:13.786592 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] MFA Rules not processed for user `2f95a367dd3148f1b31363513af8b637`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:13.788231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:13.881362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:13.953000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:13.961525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 971/1945] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:13.970223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d045b114-d720-4e7a-8cca-3caa65eda2fc service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:13.995642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:14.016090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:14.017278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:14.018060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:14.021734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:14.031265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:14.228242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 975/1946] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:07:13 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:07:14.233724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:14.254935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb857fe-3ad2-4124-bbe7-75f343ee174b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:14.399152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:14.401202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:14.401670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:14.401991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:14.408392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:14.408392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:14.531431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 972/1947] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:07:14 2022] POST /identity/v3/users => generated 298 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:07:14.536236 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:14.570907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0402bd2a-7c73-44e1-8c40-0458d6843c9b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:14.586378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:14.587039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:14.587354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:14.587593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:14.588385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:14.598316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:14.623408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 976/1948] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:07:14.625292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:14.644151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89391-9e69-4e86-b083-eee294fa6dc8 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:14.660140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:14.661108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:14.661380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:14.661657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] PATH_INFO: `/v3/projects/8e1572975eef4da5871fec165a7737d0/users/09be8a4b61024650b8e6071374b28351/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:14.693846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1572975eef4da5871fec165a7737d0, user_id=09be8a4b61024650b8e6071374b28351, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:14.700246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:14.821589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 973/1949] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:07:14 2022] PUT /identity/v3/projects/8e1572975eef4da5871fec165a7737d0/users/09be8a4b61024650b8e6071374b28351/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:07:14.825994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:14.849224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86292189-d218-4972-909c-c219c6323f59 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:15.027117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:15.029618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:15.031328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:15.031887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:15.033977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:15.041273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa5d23-4faa-4a8c-8854-cba6f8407e83 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:15.064860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 977/1950] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:07:15.066509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:15.082700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4aca21-01a5-4a4f-bb5b-5533deba73b6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:15.099143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:15.100278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:15.100775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:15.101247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] PATH_INFO: `/v3/projects/8e1572975eef4da5871fec165a7737d0/users/09be8a4b61024650b8e6071374b28351/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:15.104309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1572975eef4da5871fec165a7737d0, user_id=09be8a4b61024650b8e6071374b28351, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:15.109043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74e44c4-4be2-424f-94c9-3a4e917626d7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:15.127032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 974/1951] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:07:15 2022] PUT /identity/v3/projects/8e1572975eef4da5871fec165a7737d0/users/09be8a4b61024650b8e6071374b28351/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:07:20.841826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:20.842670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:20.842670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:20.843761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 978/1952] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:07:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:07:20.862888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:20.863519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:20.863519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:20.889968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:20.891943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:20.958574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:20.959843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:21.122755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f1200d-c8f8-46fb-a9e5-c3c7672c3a2d None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:21.129736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:21.130826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:21.131111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:21.131382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:21.134066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:21.151377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:21.154131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283950c-b7e5-46b4-9264-14258cd59728 service placement] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:21.180476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 979/1953] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:21.181975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 975/1954] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:07:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:24.843708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:24.844705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:24.858840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:24.883267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] MFA Rules not processed for user `09be8a4b61024650b8e6071374b28351`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:24.884603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:24.900735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:24.986261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 980/1955] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:24 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:29.549209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:29.551010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:29.573361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:29.640397 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8591def7-1947-4898-acd0-5670989c9c33 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:29.644211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:29.644973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:29.645248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:29.645599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:29.658248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:29.660473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0afc09c-4144-4699-957b-174f477a72da service glance] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:29.675447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 976/1956] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:29.740801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:29.772245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b3dfd6-3336-4ad4-8baa-ff7f8ef6043e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:29.784216 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:29.785160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:29.785590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:29.786002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:29.920899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:29.922370 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131ac651-ba27-4fcc-ade6-838c1861ee76 service glance] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:29.942993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 981/1957] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:30.359156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.359673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.360072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.384827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:30.386194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:30.395028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c230c-6ae6-495a-b08f-8abfe73b6942 None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.395308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c230c-6ae6-495a-b08f-8abfe73b6942 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.395587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c230c-6ae6-495a-b08f-8abfe73b6942 None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.397413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 982/1958] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:07:30.404317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.408398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.408398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.438366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] MFA Rules not processed for user `ce92d16f3ee44e9eb2a65978edc3938e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:30.439651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:30.459055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 977/1959] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:30.476101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:30.529995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 983/1960] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:30.531306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:30.534827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:30.552437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ac11-04ee-45c9-bd36-ad41bf8262b9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:30.556452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:30.557219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.557583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.557956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.566382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:30.568012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65109fea-f8c0-416b-ab0f-53c45bb8550b service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:30.580960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbeb220-fa7e-44b5-8610-2a62f65cc892 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:30.589221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 984/1961] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:30.590814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:30.591619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.592081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.592672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.603096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:30.605370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040dea6-efea-4e1c-b6a3-2f7f96b634ce service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:30.617316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 978/1962] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:07:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 21 15:07:30.670135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1634aa-626d-4e40-9e00-3a21a1f9ec6b None None] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.670503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1634aa-626d-4e40-9e00-3a21a1f9ec6b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.670881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1634aa-626d-4e40-9e00-3a21a1f9ec6b None None] PATH_INFO: `/` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.672464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 985/1963] 10.222.0.60 () {54 vars in 921 bytes} [Fri Oct 21 15:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 21 15:07:30.678075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c0604-89cb-4feb-b49c-1e9277f9a167 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.678364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c0604-89cb-4feb-b49c-1e9277f9a167 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.678660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c0604-89cb-4feb-b49c-1e9277f9a167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.705750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fa1c0604-89cb-4feb-b49c-1e9277f9a167 None None] MFA Rules not processed for user `9ee990610dd642059936212305d94fb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:30.707181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fa1c0604-89cb-4feb-b49c-1e9277f9a167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:30.718192 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 979/1964] 10.222.0.60 () {58 vars in 1018 bytes} [Fri Oct 21 15:07:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:07:30.729079 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1634aa-626d-4e40-9e00-3a21a1f9ec6b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:30.742812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1634aa-626d-4e40-9e00-3a21a1f9ec6b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:30.748039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:30.748744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:30.749025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:30.749302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:30.763487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:30.764802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:30.802430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 986/1965] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:32.742588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf604548-2c80-44d0-b256-a9c0eb624ad1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:32.743471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf604548-2c80-44d0-b256-a9c0eb624ad1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:32.743749 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf604548-2c80-44d0-b256-a9c0eb624ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:32.765678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bf604548-2c80-44d0-b256-a9c0eb624ad1 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:32.766884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bf604548-2c80-44d0-b256-a9c0eb624ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:32.779303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 980/1966] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:07:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:32.795594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:32.810539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981336d0-05e0-4397-89e3-07ec0ef8dd4f service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:32.815959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:32.816682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:32.817072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:32.817459 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:32.828068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:32.830113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:32.839636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 987/1967] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:07:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:07:36.482175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d7e8-5acd-486b-8787-a3b80da67ddf None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:36.492946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d7e8-5acd-486b-8787-a3b80da67ddf None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:36.492946 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a340d7e8-5acd-486b-8787-a3b80da67ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:36.504125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a340d7e8-5acd-486b-8787-a3b80da67ddf None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:36.505416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a340d7e8-5acd-486b-8787-a3b80da67ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:36.514999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 981/1968] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:07:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:36.532234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:36.544447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46222741-83bb-44c4-a5d0-ed5bb88d3b32 service swift] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:36.552968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:36.553592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:36.553865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:36.554332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:36.564065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:36.565641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:36.570980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:36.575713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 988/1969] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:07:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:07:37.691535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:37.691535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:37.691535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:37.720033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] MFA Rules not processed for user `f71ffc3d05d4470fa533cfd6f22184b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:37.722617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:37.799278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:37.862852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 982/1970] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:38.211112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:38.213464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:38.325217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac7803-ac9b-4b49-9f9c-605d6bfd7a20 service swift] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:38.331530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:38.332547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:38.332992 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:38.333641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] PATH_INFO: `/v3/users/6b59e9b0ec6e4fee99a93c58f9dcd6ba` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:38.335323 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b59e9b0ec6e4fee99a93c58f9dcd6ba)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:38.337617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:38.535369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b59e9b0ec6e4fee99a93c58f9dcd6ba'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:38.559389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b59e9b0ec6e4fee99a93c58f9dcd6ba'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:38.560144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b59e9b0ec6e4fee99a93c58f9dcd6ba'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:38.571300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b59e9b0ec6e4fee99a93c58f9dcd6ba'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:38.578889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d897ed24-6a11-4a8b-9007-8879b446c0df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b59e9b0ec6e4fee99a93c58f9dcd6ba'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:38.735171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 989/1971] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:07:38 2022] DELETE /identity/v3/users/6b59e9b0ec6e4fee99a93c58f9dcd6ba => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:07:39.097997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:39.116309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a2780f-37c3-40bd-8151-a74f17a73ad3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:39.218289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:39.219056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:39.219828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:39.220648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] PATH_INFO: `/v3/projects/780201e2794a47c294b91f39978cfbf4` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:39.222754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=780201e2794a47c294b91f39978cfbf4)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:39.226227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:39.387338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780201e2794a47c294b91f39978cfbf4', 'request_id': 'req-29d22a36-8046-4313-a884-2b246596fa0c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:39.534986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780201e2794a47c294b91f39978cfbf4', 'request_id': 'req-29d22a36-8046-4313-a884-2b246596fa0c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:39.542394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780201e2794a47c294b91f39978cfbf4', 'request_id': 'req-29d22a36-8046-4313-a884-2b246596fa0c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:39.556587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] The token cache is being invalidate because project 780201e2794a47c294b91f39978cfbf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:07:39.556885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:39.557593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-29d22a36-8046-4313-a884-2b246596fa0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:07:39.560410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 983/1972] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:07:39 2022] DELETE /identity/v3/projects/780201e2794a47c294b91f39978cfbf4 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:07:39.565450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:39.566141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:39.566592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:39.597285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:39.598490 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:39.690213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 990/1973] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:39.697471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:39.697903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:39.698009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:39.738140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:39.767977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:39.803761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 984/1974] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:39.807847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:39.829551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac96d71-2c71-4bb7-bc61-79fd5358b958 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:39.836891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:39.837629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:39.837981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:39.838330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:39.839172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:07:39.840608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:39.842328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:39.870376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 991/1975] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:07:39.874859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:39.887593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1d9c8a-a3d3-446e-adc1-1f91d8bcc1cf None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:39.905104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:39.905852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:39.906068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:39.906349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:39.907031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:39.908928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:40.058535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 985/1976] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:07:39 2022] POST /identity/v3/projects => generated 365 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:07:40.060468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:40.068838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:40.074713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b162a718-865b-44d0-86f9-b6cced2a89b9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:40.215522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:40.219393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:40.226068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:40.226068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:40.226068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:40.239343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:40.907404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 992/1977] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:07:40 2022] POST /identity/v3/users => generated 283 bytes in 845 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:07:40.910285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:40.938094 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e8f4f-60e8-4a17-8e5c-51802829de82 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:40.957245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:41.070492 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 986/1978] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:07:41.076476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:41.095541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965a20f-d7e7-48f2-a3eb-8a70a67d0a8c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:41.104646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:41.105541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:41.105997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:41.106432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:41.107266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:41.110867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcb96f-77e4-4acb-9f08-5965065f812f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:41.160361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 993/1979] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:07:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:07:41.163142 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:41.188278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4c2fb-5709-4d1b-b48d-b5823ee11814 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:41.208661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:41.209677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:41.210980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:41.211435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] PATH_INFO: `/v3/projects/57def3b917f4491ab891d9085e5015e8/users/34e2554d7d6c49c6888036af6a34dfdf/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:41.270546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=57def3b917f4491ab891d9085e5015e8, user_id=34e2554d7d6c49c6888036af6a34dfdf, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:41.275054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:41.390140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 987/1980] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:07:41 2022] PUT /identity/v3/projects/57def3b917f4491ab891d9085e5015e8/users/34e2554d7d6c49c6888036af6a34dfdf/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:07:41.401761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14334f-0e70-4100-8c3f-a45ea56ce072 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:41.402760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14334f-0e70-4100-8c3f-a45ea56ce072 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:41.402940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14334f-0e70-4100-8c3f-a45ea56ce072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:41.459502 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ec14334f-0e70-4100-8c3f-a45ea56ce072 None None] MFA Rules not processed for user `34e2554d7d6c49c6888036af6a34dfdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:41.460757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec14334f-0e70-4100-8c3f-a45ea56ce072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:41.695142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 994/1981] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:44.867193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:44.869168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:44.877429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:44.942682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b0864-b608-4eeb-b9ba-23afa65b19aa admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:44.946762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:44.947377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:44.947657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:44.947926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:44.962687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:44.964121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749669bc-c1d6-4b13-bc68-de5400f4b8e5 service glance] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:44.981934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 988/1982] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:07:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:07:44.990895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:44.991445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:44.991445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:45.014259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] MFA Rules not processed for user `2f95a367dd3148f1b31363513af8b637`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:45.015567 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:45.080579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 995/1983] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:07:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:07:53.125391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c773-2a90-4618-b70f-1868788bcf98 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:53.126022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c773-2a90-4618-b70f-1868788bcf98 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:53.126022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996c773-2a90-4618-b70f-1868788bcf98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:53.162205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0996c773-2a90-4618-b70f-1868788bcf98 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:07:53.169165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0996c773-2a90-4618-b70f-1868788bcf98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:53.336211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 989/1984] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:07:53.353218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:07:53.357878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:07:53.389461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:07:53.455163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a53e1-9def-416f-aec1-f48b7cf99716 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:07:53.463744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:07:53.465156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:07:53.465647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:07:53.466588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:07:53.477300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:07:53.479819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:07:53.490335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 996/1985] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:07:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:08:03.303797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=49 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:03.826861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=24 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:08.960107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:08.962313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:08.962797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:08.987084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:08:08.988294 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:08.995496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 990/1986] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:08:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:08:09.009619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:09.021295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f260cf2c-b496-451f-83eb-0c50f0fc1dea service swift] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:09.026711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:09.027320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:09.028025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:09.028025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:09.036449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:09.038192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bdda56-96bb-46ac-9e30-68fe176ca205 service swift] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:09.047483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 997/1987] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:08:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:08:18.981162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:19.044300 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=6 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:21.074263 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:21.141604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8faff40-5b36-4d2d-bca8-3fbd3a0336da None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:21.146147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:21.146820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:21.147102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:21.147376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] PATH_INFO: `/v3/users/34e2554d7d6c49c6888036af6a34dfdf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:21.148526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e2554d7d6c49c6888036af6a34dfdf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:21.150483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:21.358899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e2554d7d6c49c6888036af6a34dfdf'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:21.359772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e2554d7d6c49c6888036af6a34dfdf'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:21.376010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e2554d7d6c49c6888036af6a34dfdf'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:21.448834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e2554d7d6c49c6888036af6a34dfdf'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:21.449767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e2554d7d6c49c6888036af6a34dfdf'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:21.464353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 991/1988] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:08:21 2022] DELETE /identity/v3/users/34e2554d7d6c49c6888036af6a34dfdf => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:08:21.469656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3c4ba-79cc-46c2-8adf-bb23752bd651 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:21.470170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3c4ba-79cc-46c2-8adf-bb23752bd651 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:21.470339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3c4ba-79cc-46c2-8adf-bb23752bd651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:21.490794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c4c3c4ba-79cc-46c2-8adf-bb23752bd651 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:08:21.492101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c4c3c4ba-79cc-46c2-8adf-bb23752bd651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:21.571991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 998/1989] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:08:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:08:21.582787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:21.584368 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:21.671854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c35fb-8108-4c27-9a08-df92d6257808 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:21.674344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:21.674997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:21.675283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:21.675540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:21.684952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:21.687085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:21.698875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 992/1990] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:08:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:08:21.917573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:21.917861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:21.918138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:21.939894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] MFA Rules not processed for user `1eb3641974744d83816b6f3ad7bb6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:08:21.941327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:22.016659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 999/1991] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:08:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:08:22.131615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:22.132905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:22.153313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbac20f6-8401-43d9-8aaf-ce4c33fd622e service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:22.158693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:22.159539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:22.160052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:22.160249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] PATH_INFO: `/v3/projects/57def3b917f4491ab891d9085e5015e8` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:22.161817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57def3b917f4491ab891d9085e5015e8)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:22.163997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:22.182441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:22.295497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57def3b917f4491ab891d9085e5015e8', 'request_id': 'req-180e768b-74c9-4c0b-8deb-da06aca30a00'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:22.362876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57def3b917f4491ab891d9085e5015e8', 'request_id': 'req-180e768b-74c9-4c0b-8deb-da06aca30a00'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:22.369088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57def3b917f4491ab891d9085e5015e8', 'request_id': 'req-180e768b-74c9-4c0b-8deb-da06aca30a00'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:22.384860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] The token cache is being invalidate because project 57def3b917f4491ab891d9085e5015e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:08:22.385319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:22.386235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:08:22.402711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 993/1992] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:08:22 2022] DELETE /identity/v3/projects/57def3b917f4491ab891d9085e5015e8 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:08:24.140880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:24.142295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:24.153965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e3a816-6d76-471b-8e0a-a78b72703939 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:24.202872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:24.203546 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:24.203824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:24.204093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:24.205870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:24.271253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:24.273622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:24.275195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467b6ada-8b53-4878-8595-485f2e47176d service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:24.286866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1000/1993] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:08:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:08:32.454930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:32.503267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:08:32.504724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:32.517902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e768b-74c9-4c0b-8deb-da06aca30a00 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:08:32.521435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:08:32.522030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:32.522308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:32.522584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:32.524249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:32.586260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:08:32.587707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f6a34-95db-4901-be97-9e251e4f8708 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:08:32.597650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 994/1994] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:08:32 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:08:34.431297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:40.051345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:08:40.051960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:08:40.051960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:08:40.072398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] MFA Rules not processed for user `8632f75445ef4fd3a2f1ebbc2ed87e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:08:40.073604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:08:40.134795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1001/1995] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:08:40 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:08:42.593486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=30 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:08:50.123796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=25 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:09:10.979223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:10.979769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:10.979934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:11.000312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] MFA Rules not processed for user `4593c3fd1739432fa57f4a94c65cd299`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:09:11.002068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:11.055460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 995/1996] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:09:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:09:11.097691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:11.099142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:11.135603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:11.146297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feefca83-6044-4b5f-a4e4-d9207d4c5580 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:11.150228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:11.150866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:11.151264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:11.151542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:11.153409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:11.159176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a44b24-b878-4e11-8dac-d42d11220642 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:11.162926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:11.163548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:11.163827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:11.164105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:11.165830 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:11.238466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:11.239950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:11.246054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:11.247521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:11.252160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 996/1997] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:09:11.260711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1002/1998] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:09:21.246140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=59 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:09:21.256592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=54 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:09:22.376743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:22.378119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:22.393023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97514cf1-0854-448a-afb0-7a384659073f service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:22.398590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:22.399204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:22.399479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:22.399755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:22.401376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:22.413468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:22.415096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:22.425176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 997/1999] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:22 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:09:24.950563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:24.952366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:24.966685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156aabb2-729c-4d73-8a07-915eac778091 service glance] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:24.974818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:24.975504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:24.976151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:24.976151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:24.977884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:25.039793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:25.041397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:25.052560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1003/2000] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:09:27.380104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:27.381517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:27.438723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a040e-dc6e-4d52-9305-8c8d24db508d service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:27.442245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:27.442870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:27.443147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:27.443417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:27.445097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:27.455866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:27.457268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0989daa-c0b0-4072-8883-06e934c72846 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:27.465995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 998/2001] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:09:35.046625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=28 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:09:37.462036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=32 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:09:44.848002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:44.859919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ea72-26c4-4ee1-bd42-435cdd39af2b service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:44.865984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:44.866713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:44.867078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:44.867421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:44.869281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:44.882787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:44.884748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:44.896632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1004/2002] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:09:51.011343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:51.011863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:51.011962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:51.030918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] MFA Rules not processed for user `1eb3641974744d83816b6f3ad7bb6f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:09:51.032162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:51.039284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 999/2003] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:09:51.418386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:51.429235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5227b363-fe3b-4664-8d6f-b70a8109d620 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:51.432760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:51.433348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:51.437290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:51.437290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:51.437290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:51.445625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:51.447107 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:51.454969 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1005/2004] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:09:51.878659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:51.880864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:51.951801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8971dad4-bdbb-47b7-a410-416c78442f3e None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:51.956768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:51.958045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:51.958525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:51.958845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:51.960566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:51.971961 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:51.974244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:51.984516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1000/2005] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:09:56.072922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:09:56.074319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:56.085654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f908a-0e4d-451b-a95a-ee40125a841f service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:09:56.089418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:09:56.091296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:09:56.091575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:09:56.091853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:09:56.093679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:09:56.111089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:09:56.112775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:09:56.121817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1006/2006] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:09:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:01.980797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=35 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:02.438998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:02.448372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb8ca2-9c7e-4ab5-837b-f39508d18f72 service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:02.452059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:02.452648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:02.452927 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:02.453201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:02.454937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:02.518505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:02.519965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:02.530166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1001/2007] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:05.622755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:05.632928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=19 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:05.645405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a317ed-7c33-4716-978f-9407fb73926f service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:05.653329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:05.654233 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:05.654765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:05.655203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:05.657243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:05.673604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:05.675483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:05.694965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1007/2008] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:06.265376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:06.292223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ab82a-93d0-4fc1-8bd7-e96cab23832d service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:06.299274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:06.300481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:06.300481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:06.300724 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:06.302429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:06.315761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:06.317362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:06.329268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1002/2009] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:07.027060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:07.027060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:07.035706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac458311-5387-4a82-b1dc-bdac3f6011bb service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:07.040951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:07.042074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:07.042364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:07.042699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] PATH_INFO: `/v3/users/8632f75445ef4fd3a2f1ebbc2ed87e5b` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:07.043847 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8632f75445ef4fd3a2f1ebbc2ed87e5b)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:07.046036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:07.421082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8632f75445ef4fd3a2f1ebbc2ed87e5b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:07.445499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8632f75445ef4fd3a2f1ebbc2ed87e5b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:07.459949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8632f75445ef4fd3a2f1ebbc2ed87e5b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:07.469255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8632f75445ef4fd3a2f1ebbc2ed87e5b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:07.469956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-73fcb0f6-b029-4157-9fdd-0cd35435275f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8632f75445ef4fd3a2f1ebbc2ed87e5b'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:07.553990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1008/2010] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:07 2022] DELETE /identity/v3/users/8632f75445ef4fd3a2f1ebbc2ed87e5b => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:07.866993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:07.879451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0df15c-57af-41a6-92a7-278eebd6ba9a service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:07.970604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:07.971293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:07.971577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:07.971854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] PATH_INFO: `/v3/projects/bbced7abbedc497f86e2ea999420b2a0` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:07.973057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbced7abbedc497f86e2ea999420b2a0)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:07.975859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.092417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbced7abbedc497f86e2ea999420b2a0', 'request_id': 'req-2d93bf80-2891-4b4b-8ed6-e2443d92b191'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:08.188003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbced7abbedc497f86e2ea999420b2a0', 'request_id': 'req-2d93bf80-2891-4b4b-8ed6-e2443d92b191'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:08.195139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbced7abbedc497f86e2ea999420b2a0', 'request_id': 'req-2d93bf80-2891-4b4b-8ed6-e2443d92b191'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:08.208760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] The token cache is being invalidate because project bbced7abbedc497f86e2ea999420b2a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:08.209608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:08.212855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2d93bf80-2891-4b4b-8ed6-e2443d92b191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:08.216133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1003/2011] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:07 2022] DELETE /identity/v3/projects/bbced7abbedc497f86e2ea999420b2a0 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:08.219440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.219920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.220031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.242863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:08.244203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:08.318384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1009/2012] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:08.327081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.327081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.327407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.351596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:08.353102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:08.362135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1004/2013] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:08.367271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.378552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b861f-40b9-4b9a-8546-11ac2d3fd36c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.385238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.385865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.386178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.386578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.387303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:08.387611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.389469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.402486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1010/2014] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:08.407079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.415668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:08.420326 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022242da-f034-4efe-85d1-f753dd958ff6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.426941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.427574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.428367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.428653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.429298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.431293 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.561915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1005/2015] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:08 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:08.566141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.577075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c83dcd-a32f-4566-8bc9-a0a86f30c6bc admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.635180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:08.659763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.660398 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.660680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.660952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.661607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.664086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.786574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1011/2016] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:08 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:08.791395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.801918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e203bd-60a1-4e43-beed-67d412d1e39d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.807652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.808287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.808569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.808845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.809486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.811395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.821679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1006/2017] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:10:08.825612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.835609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f8c66e-bcb1-4434-a836-31ac73a926c6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.840481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.841115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.841392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.841659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.842289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.843894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cef9bb-c6c4-4d91-881b-85533e687aef admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.854947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1012/2018] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:08.858987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:08.869565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f33a7-9d55-4919-900a-458bbed77cd7 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:08.874171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:08.874853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.875129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.875418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] PATH_INFO: `/v3/projects/6e0b585adcbe46adb374b8989ba3c4f1/users/0b7e7a5db4054a30994e9cd5ea36109c/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:08.889912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e0b585adcbe46adb374b8989ba3c4f1, user_id=0b7e7a5db4054a30994e9cd5ea36109c, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:08.892986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:08.983299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1007/2019] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:08 2022] PUT /identity/v3/projects/6e0b585adcbe46adb374b8989ba3c4f1/users/0b7e7a5db4054a30994e9cd5ea36109c/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:08.984275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:08.984600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:08.984878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:09.016476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] MFA Rules not processed for user `0b7e7a5db4054a30994e9cd5ea36109c`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:09.017800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:09.097971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1013/2020] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:09.110158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:09.113004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:09.202308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57395af7-c8c8-4575-af40-e8c0a5470c03 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:09.208214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:09.214578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:09.214578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:09.214578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:09.218581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:09.220202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18759ab5-b505-4cd9-bc77-476dfc249b13 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:09.230067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1008/2021] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:09.503460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:09.573329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a0026-bedf-4d6c-a3a7-5d6e9dc1d8f9 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:09.579214 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:09.579726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:09.580009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:09.580282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] PATH_INFO: `/v3/users/0b7e7a5db4054a30994e9cd5ea36109c` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:09.581648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7e7a5db4054a30994e9cd5ea36109c)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:09.583411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:09.851197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7e7a5db4054a30994e9cd5ea36109c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:09.865454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7e7a5db4054a30994e9cd5ea36109c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:09.868294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7e7a5db4054a30994e9cd5ea36109c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:09.875834 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7e7a5db4054a30994e9cd5ea36109c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:09.876524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7e7a5db4054a30994e9cd5ea36109c'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:09.969212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1014/2022] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:09 2022] DELETE /identity/v3/users/0b7e7a5db4054a30994e9cd5ea36109c => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:09.975934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:09.976448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:09.976548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:09.998885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:10.000875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:10.078605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1009/2023] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:10.090810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:10.092711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:10.094005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:10.162610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d06fd4-6242-40fa-8dea-b160eb2ac04b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:10.172528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:10.183076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:10.183076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:10.183076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:10.190575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:10.191872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d4edb-da5e-4d5b-87c5-2c66bb99b049 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:10.203271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1015/2024] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:10.831115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:10.832917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:10.851401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e07064d-39f6-454a-9935-a31cc23f0e62 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:10.851401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:10.851401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:10.851683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:10.851765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] PATH_INFO: `/v3/projects/6e0b585adcbe46adb374b8989ba3c4f1` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:10.852886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e0b585adcbe46adb374b8989ba3c4f1)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:10.855660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:10.966821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:10.991840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e0b585adcbe46adb374b8989ba3c4f1', 'request_id': 'req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:11.074252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e0b585adcbe46adb374b8989ba3c4f1', 'request_id': 'req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:11.086517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e0b585adcbe46adb374b8989ba3c4f1', 'request_id': 'req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:11.096259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] The token cache is being invalidate because project 6e0b585adcbe46adb374b8989ba3c4f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:11.096690 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:11.097494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-304d7c5a-e528-4432-b5e0-a2d1a1df8aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:11.099121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1010/2025] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:10 2022] DELETE /identity/v3/projects/6e0b585adcbe46adb374b8989ba3c4f1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:11.104340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.104863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.104863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.125401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:11.126623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:11.219434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1016/2026] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:11.225204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.225896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.225987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.269693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:11.272565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:11.286692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1011/2027] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:11.288495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.298860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95fcf85-64e5-466d-a2f2-ec6913f1b8f5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.322699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.323773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.324052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.324441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.325114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:11.325445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.327482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.346683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1017/2028] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:11.349231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.363108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c6b1df-ef20-4dd4-b1cb-21ae82dff7f8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.378169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.379059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.379348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.379727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.380313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.385698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.522525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1012/2029] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:11 2022] POST /identity/v3/projects => generated 363 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:11.526838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.536955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca64230-ea9e-4e8e-8126-51588cfe3542 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.630358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.632296 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.632746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.633235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.634675 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.637086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.653307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:11.759645 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1018/2030] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:11 2022] POST /identity/v3/users => generated 282 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:11.764299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.776883 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726bf5a7-5c48-4b89-aa12-f26bd23256bd admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.782023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.782668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.782956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.783233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.783878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.785540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.797366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1013/2031] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:11.806733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.817580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d17e28-d6da-4e0b-a38f-bd6238f59978 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.824354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.825016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.825294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.825571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.826290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.827942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e5f320-9e61-4c42-b8ff-bcf9c51c901a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.840876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1019/2032] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:11.845073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:11.856903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29c7db9-e316-4c24-b9ca-a51479ded934 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:11.863315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:11.864317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.864668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.865169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] PATH_INFO: `/v3/projects/101627a21c0140ca8b05d1aaa3fd03ec/users/1ecc7cae9dc847e2a7a7f5a219502a43/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:11.885169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101627a21c0140ca8b05d1aaa3fd03ec, user_id=1ecc7cae9dc847e2a7a7f5a219502a43, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:11.888213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:11.981393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1014/2033] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:11 2022] PUT /identity/v3/projects/101627a21c0140ca8b05d1aaa3fd03ec/users/1ecc7cae9dc847e2a7a7f5a219502a43/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:11.992769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:11.993055 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:11.993356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.021613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] MFA Rules not processed for user `1ecc7cae9dc847e2a7a7f5a219502a43`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:12.022923 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:12.095518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1020/2034] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:12.100186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.164927 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de766bed-a50a-4dde-a286-0d073517b396 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.170763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.171383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.171668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.172249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.172701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.174756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.307810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1015/2035] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:12 2022] POST /identity/v3/projects => generated 363 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:12.308982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.320480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a605f-4339-4c7d-8efd-12ed3f2c72af None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.378934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.379581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.379883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.380181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.380964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.383043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.465812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1021/2036] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:12 2022] POST /identity/v3/users => generated 288 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:12.470110 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.484113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f53d8c7-b9f4-4721-a3a9-352160b818b7 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.503045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.507685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.507685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.507685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.507685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.507685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.519508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1016/2037] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:10:12.526401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.543102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be8634-4bfb-4df4-83a1-1d61d0443967 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.546770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.547518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.547994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.548404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] PATH_INFO: `/v3/projects/414d220143b942f4af18ea69e2db9bda/users/81942ededcb4430289f94b04676442f6/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.569519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] RBAC: Authorizing `identity:create_grant(project_id=414d220143b942f4af18ea69e2db9bda, user_id=81942ededcb4430289f94b04676442f6, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.572987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.746545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1022/2038] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:12 2022] PUT /identity/v3/projects/414d220143b942f4af18ea69e2db9bda/users/81942ededcb4430289f94b04676442f6/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:12.750608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.761450 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32f32-4cb8-44fb-b81b-4a7836348dd3 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.797708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:12.865658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.870364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.870662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.871031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.871703 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.875486 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f3eee-03ab-4bc3-b82d-9859ba7d8220 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.906008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:12.907213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1017/2039] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:12.922464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30e8b5-6a4e-4432-a746-51c4ed94357f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:12.937927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:12.939002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.939387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.939827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] PATH_INFO: `/v3/projects/414d220143b942f4af18ea69e2db9bda/users/81942ededcb4430289f94b04676442f6/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:12.945665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=414d220143b942f4af18ea69e2db9bda, user_id=81942ededcb4430289f94b04676442f6, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:12.946247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:12.966681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1023/2040] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:12 2022] PUT /identity/v3/projects/414d220143b942f4af18ea69e2db9bda/users/81942ededcb4430289f94b04676442f6/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:12.981269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:12.981569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:12.982313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:13.020768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] MFA Rules not processed for user `81942ededcb4430289f94b04676442f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:13.023511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:13.148694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1018/2041] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:13.157696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:13.159458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:13.195507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:13.237331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb2372-30b5-4389-8f2d-a25bbc392b0f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:13.242110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:13.243406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:13.243994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:13.244322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:13.257880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:13.259525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ae6e8-c195-4dc5-893a-95b6c021e389 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:13.271408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1024/2042] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:13.525185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:13.540855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99708bfb-48b8-4127-985d-c6a5f0c768a8 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:13.547011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:13.547708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:13.548033 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:13.548283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:13.625173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:13.627140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:13.639370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1019/2043] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:16.684700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698581d-8db6-493b-ad26-bd2d0b09bbd7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:16.685210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698581d-8db6-493b-ad26-bd2d0b09bbd7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:16.685365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698581d-8db6-493b-ad26-bd2d0b09bbd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:16.706841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c698581d-8db6-493b-ad26-bd2d0b09bbd7 None None] MFA Rules not processed for user `3418ee58bd51418d9594b98ed594f506`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:16.709576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c698581d-8db6-493b-ad26-bd2d0b09bbd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:16.770294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1025/2044] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:17.124198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:17.125664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:17.179558 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:17.184696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90d0773-86d4-4abe-b2db-c066bd5ee775 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:17.188395 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:17.192149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:17.192149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:17.192149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:17.192149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:17.201698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:17.203156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:17.211623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1020/2045] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:10:18.224863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:18.225352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:18.225451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:18.249536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:18.249536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:18.303453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1026/2046] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:10:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:18.439926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:18.441540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:18.515000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f68f1-70c5-49db-9a9c-9b6b7ff7022e service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:18.523502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:18.524127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:18.524691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:18.524890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:18.533684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:18.535346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f200852-9e18-4f01-8972-1792ffb6a770 service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:18.545014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1021/2047] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:10:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:10:25.387110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:25.496218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64f568-a58c-4e90-85fa-23cc394bee07 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:25.502395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:25.503812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:25.504103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:25.504382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] PATH_INFO: `/v3/users/1ecc7cae9dc847e2a7a7f5a219502a43` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:25.505565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ecc7cae9dc847e2a7a7f5a219502a43)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:25.507759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:25.540510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:25.742892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ecc7cae9dc847e2a7a7f5a219502a43'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:25.756484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ecc7cae9dc847e2a7a7f5a219502a43'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:25.757876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ecc7cae9dc847e2a7a7f5a219502a43'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:25.765082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ecc7cae9dc847e2a7a7f5a219502a43'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:25.770706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ecc7cae9dc847e2a7a7f5a219502a43'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:25.869196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1027/2048] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:25 2022] DELETE /identity/v3/users/1ecc7cae9dc847e2a7a7f5a219502a43 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:25.874682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:25.875559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:25.876061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:25.897570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:25.899796 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:26.015307 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1022/2049] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:10:26.036117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:26.037757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:26.141976 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d6724-9d70-49ea-811f-03918806370a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:26.145851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172be-694e-4294-8c97-f29386304188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:26.146495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172be-694e-4294-8c97-f29386304188 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:26.146950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172be-694e-4294-8c97-f29386304188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:26.147295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873172be-694e-4294-8c97-f29386304188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:26.158532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873172be-694e-4294-8c97-f29386304188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:26.163101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873172be-694e-4294-8c97-f29386304188 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:26.175978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1028/2050] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:26.724092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:26.725191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:26.744625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbbe4b1-b78d-44c1-9bde-5db2e4c497a2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:26.768579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:26.769305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:26.769586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:26.769943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] PATH_INFO: `/v3/projects/101627a21c0140ca8b05d1aaa3fd03ec` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:26.771113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=101627a21c0140ca8b05d1aaa3fd03ec)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:26.773746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:26.940112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101627a21c0140ca8b05d1aaa3fd03ec', 'request_id': 'req-f030beea-0434-48ef-9b94-0d1b36177e4b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.033616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101627a21c0140ca8b05d1aaa3fd03ec', 'request_id': 'req-f030beea-0434-48ef-9b94-0d1b36177e4b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.042931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101627a21c0140ca8b05d1aaa3fd03ec', 'request_id': 'req-f030beea-0434-48ef-9b94-0d1b36177e4b'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.064900 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] The token cache is being invalidate because project 101627a21c0140ca8b05d1aaa3fd03ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:27.065330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.066138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.067955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1023/2051] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:26 2022] DELETE /identity/v3/projects/101627a21c0140ca8b05d1aaa3fd03ec => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:27.072919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172be-694e-4294-8c97-f29386304188 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:27.084257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873172be-694e-4294-8c97-f29386304188 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:27.161287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:27.161948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:27.162230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:27.162507 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] PATH_INFO: `/v3/users/81942ededcb4430289f94b04676442f6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:27.164438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81942ededcb4430289f94b04676442f6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:27.167008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:27.391076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:27.394009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81942ededcb4430289f94b04676442f6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.406578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81942ededcb4430289f94b04676442f6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.407355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81942ededcb4430289f94b04676442f6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.415202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81942ededcb4430289f94b04676442f6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.415856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-04367a19-54dd-4423-9124-e7866556500f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81942ededcb4430289f94b04676442f6'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:27.483835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1029/2052] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:27 2022] DELETE /identity/v3/users/81942ededcb4430289f94b04676442f6 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:28.052511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.054820 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:28.068055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:28.074338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f030beea-0434-48ef-9b94-0d1b36177e4b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.141914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.142562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.142875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.143146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] PATH_INFO: `/v3/projects/414d220143b942f4af18ea69e2db9bda` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.144271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414d220143b942f4af18ea69e2db9bda)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.146170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.277661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414d220143b942f4af18ea69e2db9bda', 'request_id': 'req-cad5a40b-ee4d-4636-a03a-2886f180aaa7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:28.353870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414d220143b942f4af18ea69e2db9bda', 'request_id': 'req-cad5a40b-ee4d-4636-a03a-2886f180aaa7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:28.370506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414d220143b942f4af18ea69e2db9bda', 'request_id': 'req-cad5a40b-ee4d-4636-a03a-2886f180aaa7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:28.387144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] The token cache is being invalidate because project 414d220143b942f4af18ea69e2db9bda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:28.387824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:28.388998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cad5a40b-ee4d-4636-a03a-2886f180aaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:28.391203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1024/2053] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:28 2022] DELETE /identity/v3/projects/414d220143b942f4af18ea69e2db9bda => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:28.396284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.396786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.396882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.426280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:28.427755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:28.520863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1030/2054] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:28.530335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.530769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.531135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.552513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:28.554514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:28.565819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1025/2055] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:28.573688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.586040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec95b4a9-3994-4436-b0cf-d9672690ac18 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.591654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.591654 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.591866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.592134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.592764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:28.593065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.594723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.608716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1031/2056] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:28.623531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.633825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca58d7be-36cd-4d3d-826a-8371c23dc301 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.638542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.639151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.639426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.639698 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.643375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.643375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.725712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1026/2057] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:28 2022] POST /identity/v3/projects => generated 387 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:28.730315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.740794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609dd4c6-80d2-4e20-883c-e6833f669a4d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.802528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.803315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.803607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.803879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.805121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.807076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.916594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1032/2058] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:28 2022] POST /identity/v3/users => generated 294 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:28.917734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.928346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da582c48-7dc6-4c28-baae-e877079d2db0 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.933547 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.934153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.934438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.937688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.937688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.937688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.946984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1027/2059] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:28.951213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.961453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfeeed1-b59b-475e-b6d4-27da07287991 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:28.965988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:28.966574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:28.966884 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:28.967148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:28.967791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:28.969351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:28.980727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1033/2060] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:28.984821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:28.996267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059422-8ebc-4219-8d79-b5bf5e3783d4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.002357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.003495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.003999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.004321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] PATH_INFO: `/v3/projects/2710c96bbd804230b5f28e9390c2e5e9/users/208f1d6a694f4e0e94f88374ed04b57e/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.012736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.015261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:29.021286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2710c96bbd804230b5f28e9390c2e5e9, user_id=208f1d6a694f4e0e94f88374ed04b57e, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.025141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f62026b-866e-4006-bbb6-d73efc0ef350 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.030129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb20cc57-609e-4672-b948-3e52a33789b1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.035123 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.035754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.036033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.036303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] PATH_INFO: `/v3/users/2f95a367dd3148f1b31363513af8b637` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.037436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f95a367dd3148f1b31363513af8b637)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.040178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.062961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:29.106623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1028/2061] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:28 2022] PUT /identity/v3/projects/2710c96bbd804230b5f28e9390c2e5e9/users/208f1d6a694f4e0e94f88374ed04b57e/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:29.117844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.118066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.118341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.147318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] MFA Rules not processed for user `208f1d6a694f4e0e94f88374ed04b57e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:29.149107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:29.168199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:29.216373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f95a367dd3148f1b31363513af8b637'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.226536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1029/2062] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:10:29.231620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.237469 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f95a367dd3148f1b31363513af8b637'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.238188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f95a367dd3148f1b31363513af8b637'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.246568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f95a367dd3148f1b31363513af8b637'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.247684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f95a367dd3148f1b31363513af8b637'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.298911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ad7e4-93cb-40fb-816f-c5a3a4391276 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.303680 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.304477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.304850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.305131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.305766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.307708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.315182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1034/2063] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:29 2022] DELETE /identity/v3/users/2f95a367dd3148f1b31363513af8b637 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:29.404855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1030/2064] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:29 2022] POST /identity/v3/projects => generated 387 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:29.405977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.407539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:29.484633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbcda4-485e-48d4-a7ff-87baa57331f9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.545930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.546692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.546979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.547255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.547902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.549833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.618891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.620927 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:29.638891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a001db21-6a3d-4713-b0c8-288fb2d7635b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.646385 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.647380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.647701 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.648646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] PATH_INFO: `/v3/projects/a77531edf7224da8847bb5c9813ae7da` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.650586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a77531edf7224da8847bb5c9813ae7da)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.658118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.658695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1035/2065] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:29 2022] POST /identity/v3/users => generated 300 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:29.661512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.674116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48708d20-528c-43f7-9fc6-92f07ccd4e2e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.679498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.683357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.683357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.683357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.683357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.683357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.693511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1036/2066] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:29.698456 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.708139 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a343790-b0f5-4a18-b4c2-3e53df024dd1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.713676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.714330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.714605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.714936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] PATH_INFO: `/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc/users/a11ae8d9cb674ba996bc6cb126d170f0/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.730308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc2b22579864c82868cf8c8dcbfddcc, user_id=a11ae8d9cb674ba996bc6cb126d170f0, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.733002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.796067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=7,keystone:SELECT=88,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:29.803110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1037/2067] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:29 2022] PUT /identity/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc/users/a11ae8d9cb674ba996bc6cb126d170f0/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:29.807250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.814044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a77531edf7224da8847bb5c9813ae7da', 'request_id': 'req-effae1b6-163a-4b1d-9577-87f4f82cc221'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.872863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff71b9bc-e980-43f8-b74b-8811481aa126 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.931281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a77531edf7224da8847bb5c9813ae7da', 'request_id': 'req-effae1b6-163a-4b1d-9577-87f4f82cc221'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.932536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.933159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.933457 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.933716 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.934391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.936064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.939706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a77531edf7224da8847bb5c9813ae7da', 'request_id': 'req-effae1b6-163a-4b1d-9577-87f4f82cc221'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.948826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1038/2068] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:10:29.952346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.964043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] The token cache is being invalidate because project a77531edf7224da8847bb5c9813ae7da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:29.964476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.964656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f081d7-4df2-4184-983e-5d0b51acfbf7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.965747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:29.967559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1031/2069] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:29 2022] DELETE /identity/v3/projects/a77531edf7224da8847bb5c9813ae7da => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:29.969979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:29.977910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:29.977910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:29.977910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] PATH_INFO: `/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc/users/a11ae8d9cb674ba996bc6cb126d170f0/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:29.978201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:29.978201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:29.978366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc2b22579864c82868cf8c8dcbfddcc, user_id=a11ae8d9cb674ba996bc6cb126d170f0, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:29.978366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb5995a-14f6-43e6-9197-d443c4e2fd8e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:29.990269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effae1b6-163a-4b1d-9577-87f4f82cc221 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:29.999462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1039/2070] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:29 2022] PUT /identity/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc/users/a11ae8d9cb674ba996bc6cb126d170f0/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:30.001517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.001808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.002082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.024910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:30.034346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] MFA Rules not processed for user `a11ae8d9cb674ba996bc6cb126d170f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:30.036705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.061745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:30.069282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.069282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.069282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] PATH_INFO: `/v3/users/09be8a4b61024650b8e6071374b28351` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.069282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] RBAC: Authorizing `identity:delete_user(user_id=09be8a4b61024650b8e6071374b28351)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:30.069282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:30.114789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1040/2071] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:30.127308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:30.128829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.147374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:30.187238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a0865e-6eef-47b7-89a1-6a41c1879470 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:30.190990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:30.191596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.191876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.192143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.201993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:30.203453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:30.213335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1041/2072] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:10:30.243906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:30.245346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.258782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d144ec-ffdc-426a-971c-d9071edcdbd6 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:30.263425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:30.264062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.264340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.264612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] PATH_INFO: `/v3/users/208f1d6a694f4e0e94f88374ed04b57e` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.265738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=208f1d6a694f4e0e94f88374ed04b57e)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:30.267708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:30.315688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09be8a4b61024650b8e6071374b28351'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.316342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09be8a4b61024650b8e6071374b28351'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.328798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09be8a4b61024650b8e6071374b28351'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.456451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09be8a4b61024650b8e6071374b28351'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.457051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12928c55-7dae-45a4-a55f-50a2c58e104b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09be8a4b61024650b8e6071374b28351'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.470203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1032/2073] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:29 2022] DELETE /identity/v3/users/09be8a4b61024650b8e6071374b28351 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:30.500696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '208f1d6a694f4e0e94f88374ed04b57e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.512707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '208f1d6a694f4e0e94f88374ed04b57e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.513362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '208f1d6a694f4e0e94f88374ed04b57e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.520458 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '208f1d6a694f4e0e94f88374ed04b57e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.521142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '208f1d6a694f4e0e94f88374ed04b57e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:30.583953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1042/2074] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:30 2022] DELETE /identity/v3/users/208f1d6a694f4e0e94f88374ed04b57e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:30.586901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.587420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.587882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.620088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:30.622108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.689009 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1033/2075] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:30.699015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:30.700421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.759953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60c2eb-d254-4357-8550-74e364db89bc admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:30.763494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:30.764097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.764374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.764644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.774877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:30.776386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:30.785797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1043/2076] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:30.796077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:30.798621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:30.830505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f243b7c-30fd-47ff-abf0-35231f695c52 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:30.836968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:30.837899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:30.838318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:30.838597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] PATH_INFO: `/v3/projects/8e1572975eef4da5871fec165a7737d0` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:30.840299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1572975eef4da5871fec165a7737d0)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:30.842460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:30.942516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:30.949163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1572975eef4da5871fec165a7737d0', 'request_id': 'req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.010506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1572975eef4da5871fec165a7737d0', 'request_id': 'req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.016937 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1572975eef4da5871fec165a7737d0', 'request_id': 'req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.031312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] The token cache is being invalidate because project 8e1572975eef4da5871fec165a7737d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:31.031709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.032496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-ab1b9d03-fab1-4dd8-b5be-78898cfeddd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.047321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1034/2077] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:30 2022] DELETE /identity/v3/projects/8e1572975eef4da5871fec165a7737d0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:31.254453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:31.256548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:31.291600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96eabe4-adea-48d5-9efe-69f776cd9d97 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:31.325842 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:31.352711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:31.353417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:31.353688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:31.353962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] PATH_INFO: `/v3/projects/2710c96bbd804230b5f28e9390c2e5e9` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:31.355119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2710c96bbd804230b5f28e9390c2e5e9)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:31.357018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:31.485475 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2710c96bbd804230b5f28e9390c2e5e9', 'request_id': 'req-2ea4ab95-8162-4ea1-acf9-7ac08502a832'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.573043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2710c96bbd804230b5f28e9390c2e5e9', 'request_id': 'req-2ea4ab95-8162-4ea1-acf9-7ac08502a832'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.581208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2710c96bbd804230b5f28e9390c2e5e9', 'request_id': 'req-2ea4ab95-8162-4ea1-acf9-7ac08502a832'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.587500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:31.587982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:31.588065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:31.593803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] The token cache is being invalidate because project 2710c96bbd804230b5f28e9390c2e5e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:31.594250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.594934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.596626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1044/2078] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:31 2022] DELETE /identity/v3/projects/2710c96bbd804230b5f28e9390c2e5e9 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:31.600340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:31.606911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:31.618833 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:31.620140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:31.679622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4ab95-8162-4ea1-acf9-7ac08502a832 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:31.684921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:31.685552 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:31.685832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:31.686104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] PATH_INFO: `/v3/users/a11ae8d9cb674ba996bc6cb126d170f0` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:31.687270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11ae8d9cb674ba996bc6cb126d170f0)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:31.689175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:31.698112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1035/2079] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:10:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:31.710458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:31.712321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:31.782964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b40532-57f9-4456-aed7-8c72daae6e0a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:31.788097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:31.789273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:31.789528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:31.789782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:31.798507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:31.800139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:31.809304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1036/2080] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:10:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 21 15:10:31.863485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11ae8d9cb674ba996bc6cb126d170f0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.876628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11ae8d9cb674ba996bc6cb126d170f0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.877413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11ae8d9cb674ba996bc6cb126d170f0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.878704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:31.885056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11ae8d9cb674ba996bc6cb126d170f0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.885726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-9eebae2f-9d07-4665-9438-dd2291930418 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11ae8d9cb674ba996bc6cb126d170f0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:31.989486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1045/2081] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:31 2022] DELETE /identity/v3/users/a11ae8d9cb674ba996bc6cb126d170f0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:32.633219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:32.654178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c2e12-f5b3-4c3b-b269-98dcc52ea29e service swift] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:32.741441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:32.742943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:32.743319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:32.743692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] PATH_INFO: `/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:32.744972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc2b22579864c82868cf8c8dcbfddcc)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:32.747308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:32.754806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:32.864483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc2b22579864c82868cf8c8dcbfddcc', 'request_id': 'req-1b730c8b-22b8-4275-8070-24264bf74ed7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:32.959303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc2b22579864c82868cf8c8dcbfddcc', 'request_id': 'req-1b730c8b-22b8-4275-8070-24264bf74ed7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:32.966464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc2b22579864c82868cf8c8dcbfddcc', 'request_id': 'req-1b730c8b-22b8-4275-8070-24264bf74ed7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:32.982113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] The token cache is being invalidate because project 6fc2b22579864c82868cf8c8dcbfddcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:32.982338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:32.982991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-1b730c8b-22b8-4275-8070-24264bf74ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:32.987247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1037/2082] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:32 2022] DELETE /identity/v3/projects/6fc2b22579864c82868cf8c8dcbfddcc => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:32.989589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:32.989987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:32.990167 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.011941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:33.013169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:33.091171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1046/2083] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:33.098915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.099414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.099689 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.122568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:33.123878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:33.137137 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1038/2084] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:33.138069 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.149147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76003493-0fba-4bf9-81ba-39ac881810c5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.157568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.158192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.158468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.158892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.159553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:33.159858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.161465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:33.175897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1047/2085] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:33.180653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.190264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176bfed-27f5-43e4-af73-4e4698a084b5 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.197947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.197947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.197947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.197947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.198833 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.199780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:33.292782 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1039/2086] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:33 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:33.297146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.310109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25989e8-557c-4114-919b-5e91c9e74131 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.400376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.403493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.404177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.404177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.404894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.407063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:33.526247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1048/2087] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:33 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:33.530613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.544935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151213a5-e581-4ebc-b867-ad0a3f3cae66 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.550245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.550887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.551168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.551437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.552101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.553713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:33.569369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1040/2088] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:33.570868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.581722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addc4690-39b0-4ab0-9721-5a02b3cdeee1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.588226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.588836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.590510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.590510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.590510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.594180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136e8b5-77d5-4736-b809-29c63cc075d4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:33.614701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1049/2089] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:10:33.621277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:33.650953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc147b6-a9d8-4f5e-aa43-ba884686d743 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:33.668504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:33.669456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:33.670038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:33.670377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] PATH_INFO: `/v3/projects/4c7e020e69dd401b816f57a63da04333/users/3818e7e65d674fe68d11bacde049e9eb/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:33.704545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c7e020e69dd401b816f57a63da04333, user_id=3818e7e65d674fe68d11bacde049e9eb, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:33.878824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:34.232226 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1041/2090] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:33 2022] PUT /identity/v3/projects/4c7e020e69dd401b816f57a63da04333/users/3818e7e65d674fe68d11bacde049e9eb/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:34.245400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:34.248212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:34.248212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:34.303004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] MFA Rules not processed for user `3818e7e65d674fe68d11bacde049e9eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:34.304396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:34.370988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:34.407658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1050/2091] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:34.410398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:34.495530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2c1a0-4f70-4184-8cb0-78b872bc7388 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:34.501777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:34.503100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:34.503469 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:34.503815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:34.504821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:34.507273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:34.655114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1042/2092] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:34 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:34.659310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:34.675316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc803f8e-ef9b-4ecf-998a-6721b40a06de None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:34.755819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:34.756558 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:34.756840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:34.758046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:34.758956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:34.768435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:34.858845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1051/2093] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:34 2022] POST /identity/v3/users => generated 300 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:34.863082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:34.877636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059505-2ae4-45e9-929b-061130338e50 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:34.883363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:34.884079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:34.884423 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:34.884756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:34.888012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:34.888012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:34.902348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1043/2094] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:34.907783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:34.928749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821363a-2cfb-4fe4-a608-48dce6356749 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:34.937037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:34.938356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:34.938773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:34.939057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] PATH_INFO: `/v3/projects/c708efa6db9d4c18a78547166b0c3515/users/4d8a05e457b34dffbf98db76edb00a12/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:34.961497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c708efa6db9d4c18a78547166b0c3515, user_id=4d8a05e457b34dffbf98db76edb00a12, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:34.966289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:35.047126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1052/2095] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:34 2022] PUT /identity/v3/projects/c708efa6db9d4c18a78547166b0c3515/users/4d8a05e457b34dffbf98db76edb00a12/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:35.051546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:35.059467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:35.063828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0453e-daf3-40e9-94a8-332a56652000 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:35.142911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:35.145341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:35.145672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:35.146530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:35.147952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:35.150539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281eb069-c3f3-4730-a461-1a86577c06ff admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:35.168519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1044/2096] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:35.175378 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:35.190014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4740082-f1f3-4aed-b990-f6d467932d60 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:35.196467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:35.197142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:35.197429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:35.197707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] PATH_INFO: `/v3/projects/c708efa6db9d4c18a78547166b0c3515/users/4d8a05e457b34dffbf98db76edb00a12/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:35.199709 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c708efa6db9d4c18a78547166b0c3515, user_id=4d8a05e457b34dffbf98db76edb00a12, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:35.202776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:35.215752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1053/2097] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:35 2022] PUT /identity/v3/projects/c708efa6db9d4c18a78547166b0c3515/users/4d8a05e457b34dffbf98db76edb00a12/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:35.235002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:35.235297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:35.235578 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:35.270532 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] MFA Rules not processed for user `4d8a05e457b34dffbf98db76edb00a12`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:35.271909 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:35.373222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1045/2098] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:35.388519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:35.393896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:35.481551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc654f-8d96-4b3b-8d03-56c421f0e021 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:35.486555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:35.487480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:35.487861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:35.488234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:35.517275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:35.519032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7c949-05e5-4072-bb32-1455936e93f9 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:35.544106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1054/2099] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:10:35.627008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:35.740827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cddcd1e-761a-4903-a746-51db31de2184 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:35.740827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:35.742032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:35.742570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:35.742976 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] PATH_INFO: `/v3/users/3818e7e65d674fe68d11bacde049e9eb` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:35.755624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3818e7e65d674fe68d11bacde049e9eb)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:35.757957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:35.874500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:35.967866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3818e7e65d674fe68d11bacde049e9eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:35.968677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3818e7e65d674fe68d11bacde049e9eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:35.994888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3818e7e65d674fe68d11bacde049e9eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:36.096350 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3818e7e65d674fe68d11bacde049e9eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:36.097031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3818e7e65d674fe68d11bacde049e9eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:36.115714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1046/2100] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:35 2022] DELETE /identity/v3/users/3818e7e65d674fe68d11bacde049e9eb => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:36.120784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:36.121102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:36.121402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:36.145826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:36.147098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:36.181881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:36.220609 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1055/2101] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:36.232412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:36.234676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:36.318844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40b393e-1dca-4243-a691-57cc6d4b5c84 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:36.323824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:36.324141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:36.324513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:36.324884 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:36.334959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:36.336807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:36.359488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1047/2102] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:36.828730 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:36.830749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:36.851715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f575553-12e9-4da2-a13e-6bd016db1c42 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:36.861355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:36.862274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:36.862680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:36.863076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] PATH_INFO: `/v3/projects/4c7e020e69dd401b816f57a63da04333` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:36.865449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c7e020e69dd401b816f57a63da04333)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:36.868147 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:37.094870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c7e020e69dd401b816f57a63da04333', 'request_id': 'req-c4b43c15-4776-4f95-8d3c-2de24b585e48'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.200014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c7e020e69dd401b816f57a63da04333', 'request_id': 'req-c4b43c15-4776-4f95-8d3c-2de24b585e48'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.212310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c7e020e69dd401b816f57a63da04333', 'request_id': 'req-c4b43c15-4776-4f95-8d3c-2de24b585e48'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.231208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] The token cache is being invalidate because project 4c7e020e69dd401b816f57a63da04333 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:37.231958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.232650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.238597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:37.239710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1056/2103] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:36 2022] DELETE /identity/v3/projects/4c7e020e69dd401b816f57a63da04333 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:37.251891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d2f-f452-4e44-a2ac-6e7a7442afd7 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:37.347274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:37.348664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:37.349125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:37.350237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] PATH_INFO: `/v3/users/4d8a05e457b34dffbf98db76edb00a12` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:37.353400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d8a05e457b34dffbf98db76edb00a12)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:37.355851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:37.583444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d8a05e457b34dffbf98db76edb00a12'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.587785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d8a05e457b34dffbf98db76edb00a12'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.590091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:37.611135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d8a05e457b34dffbf98db76edb00a12'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.723948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d8a05e457b34dffbf98db76edb00a12'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.723948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-73ac9042-e5bd-4ede-bb3a-e5e0efd82acf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d8a05e457b34dffbf98db76edb00a12'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:37.728112 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1048/2104] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:37 2022] DELETE /identity/v3/users/4d8a05e457b34dffbf98db76edb00a12 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:38.384956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:38.384956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:38.407442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b43c15-4776-4f95-8d3c-2de24b585e48 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:38.487772 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:38.489253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:38.489753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:38.490253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] PATH_INFO: `/v3/projects/c708efa6db9d4c18a78547166b0c3515` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:38.492586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c708efa6db9d4c18a78547166b0c3515)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:38.495943 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:38.633156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c708efa6db9d4c18a78547166b0c3515', 'request_id': 'req-1d4ee898-81de-4424-8d64-33ef9495c802'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:38.707993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c708efa6db9d4c18a78547166b0c3515', 'request_id': 'req-1d4ee898-81de-4424-8d64-33ef9495c802'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:38.712092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:38.718162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c708efa6db9d4c18a78547166b0c3515', 'request_id': 'req-1d4ee898-81de-4424-8d64-33ef9495c802'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:38.731344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] The token cache is being invalidate because project c708efa6db9d4c18a78547166b0c3515 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:38.731639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:38.735235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d4ee898-81de-4424-8d64-33ef9495c802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:38.736935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1057/2105] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:38 2022] DELETE /identity/v3/projects/c708efa6db9d4c18a78547166b0c3515 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:38.742153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:38.742777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:38.742938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:38.765426 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:38.766719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:38.840936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1049/2106] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:10:38.860274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:38.860587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:38.860824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:38.899920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:38.901545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:39.121764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1058/2107] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:39.121876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.121876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf5e275-75b9-4765-ba75-d4f5d5a20847 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.121876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.121876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.122302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1050/2108] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:39.122863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.122863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c818e-8e97-4801-9304-d1cc7edc8162 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.122863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.122863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.123238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.123238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.123238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.123238 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.440331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1059/2109] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:38 2022] POST /identity/v3/projects => generated 385 bytes in 476 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:39.444076 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.461229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa3bf4c-f084-4480-b4c3-6dbcd0291e1f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.573111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.574039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.574526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.574838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.575482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.577681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.659324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1051/2110] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:39 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:39.663871 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.678053 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd498bc0-3c31-4453-b962-9ed4fa888ca2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.684145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.684795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.685143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.685428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.686179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.690888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.704089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1060/2111] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:39.707647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.720834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66881527-17d4-4483-b2b4-521a642dc820 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.727119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.727733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.728194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.728305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.729056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.730980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ead57-eacc-44f6-b8be-2162c9ccd787 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.749281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1052/2112] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:39.752687 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:39.767785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc8ca2-89bf-4c90-bb82-0804cbe91e27 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:39.773930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:39.775477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.776648 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.777010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] PATH_INFO: `/v3/projects/9fd1bdf287b3429b9c1a69034b74d149/users/08ed74ad292841aab9d36d54e5a189c7/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.796420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd1bdf287b3429b9c1a69034b74d149, user_id=08ed74ad292841aab9d36d54e5a189c7, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:39.799317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:39.909050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1061/2113] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:39 2022] PUT /identity/v3/projects/9fd1bdf287b3429b9c1a69034b74d149/users/08ed74ad292841aab9d36d54e5a189c7/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:39.920791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:39.921083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:39.921525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:39.971008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] MFA Rules not processed for user `08ed74ad292841aab9d36d54e5a189c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:39.974779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:40.013188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:40.096975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1053/2114] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:40.098174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:40.243826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc470d1-4e04-4295-8332-8fc43034d9c2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:40.250256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:40.251126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:40.251534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:40.251914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:40.252801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:40.255190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:40.358736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1062/2115] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:40 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:40.362621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:40.378443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38e3c3-9eee-4f98-9eb4-1dffcef0ad8c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:40.480479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:40.481483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:40.481764 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:40.482032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:40.482785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:40.484664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:40.607910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:40.617731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1054/2116] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:40 2022] POST /identity/v3/users => generated 298 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:40.647048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d83bcd-65f7-4f0e-ad3e-345926c7b3a0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:40.687468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:40.691082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:40.691476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:40.691854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:40.694345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:40.701890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:40.754948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:40.768185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1063/2117] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:40.770138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8aa1ca-b42c-4f75-adb9-67c5a79e076b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:40.787061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:40.787983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:40.792953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:40.792953 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] PATH_INFO: `/v3/projects/5acd30143242435aa70986d0d9e0d4ad/users/d325a72c97c545b582a21c94c8ec7402/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:40.810939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acd30143242435aa70986d0d9e0d4ad, user_id=d325a72c97c545b582a21c94c8ec7402, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:40.814118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:40.894810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1055/2118] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:40 2022] PUT /identity/v3/projects/5acd30143242435aa70986d0d9e0d4ad/users/d325a72c97c545b582a21c94c8ec7402/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:40.899644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:40.910472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9aa3a4-9e2f-4313-a16d-9ca22c3422e3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:41.002738 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:41.010817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:41.011240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:41.011519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:41.011789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:41.012485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:41.014078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83498151-c1f9-476a-b241-c4d5ef94a0d6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:41.025731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1064/2119] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:41.029717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:41.041171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87605020-6ddf-4a31-8aac-346ad47b298a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:41.046359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:41.047059 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:41.047344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:41.047620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] PATH_INFO: `/v3/projects/5acd30143242435aa70986d0d9e0d4ad/users/d325a72c97c545b582a21c94c8ec7402/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:41.049984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acd30143242435aa70986d0d9e0d4ad, user_id=d325a72c97c545b582a21c94c8ec7402, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:41.052358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:41.073712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1056/2120] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:41 2022] PUT /identity/v3/projects/5acd30143242435aa70986d0d9e0d4ad/users/d325a72c97c545b582a21c94c8ec7402/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:41.083977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806949d7-246b-480f-a3fd-236a9a50d373 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:41.084264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806949d7-246b-480f-a3fd-236a9a50d373 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:41.084539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806949d7-246b-480f-a3fd-236a9a50d373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:41.124008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-806949d7-246b-480f-a3fd-236a9a50d373 None None] MFA Rules not processed for user `d325a72c97c545b582a21c94c8ec7402`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:41.125444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-806949d7-246b-480f-a3fd-236a9a50d373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:41.220050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1065/2121] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:41.232485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:41.237865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:41.399496 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed609af-872e-4ce2-9b84-1c26d20ef92a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:41.404596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:41.405286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:41.405662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:41.405945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:41.457919 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:41.502677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:41.508400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:41.529742 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1057/2122] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:42.462479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:42.463368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:42.463817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:42.498150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] MFA Rules not processed for user `f71ffc3d05d4470fa533cfd6f22184b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:42.499416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:42.565748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1066/2123] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:47.185977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:47.196046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7422ae8c-9416-498c-8071-a57ed786167e service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:47.199727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:47.200430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:47.201008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:47.201285 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:47.210107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:47.211843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f577f2f-9169-4c8f-afe8-a80898e435ff service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:47.221160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1058/2124] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:52.557504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=62 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:53.392270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:53.455939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59b8dfb-79c2-4489-91a2-8d478b2e4c4e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:53.461761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:53.462473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:53.462802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:53.463081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] PATH_INFO: `/v3/users/08ed74ad292841aab9d36d54e5a189c7` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:53.464885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08ed74ad292841aab9d36d54e5a189c7)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:53.466954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:53.606303 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:53.606801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:53.606893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:53.626499 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:53.627711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:53.683316 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1059/2125] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:53.693319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08ed74ad292841aab9d36d54e5a189c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:53.695838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:53.697274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:53.707256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08ed74ad292841aab9d36d54e5a189c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:53.707990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08ed74ad292841aab9d36d54e5a189c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:53.716071 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08ed74ad292841aab9d36d54e5a189c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:53.716779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08ed74ad292841aab9d36d54e5a189c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:53.766551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd227dd7-cc31-4119-9377-46c08df7fafc None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:53.771625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:53.772274 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:53.772535 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:53.772809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:53.775548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:53.786229 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:53.787846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5044f282-bf80-4564-bd04-3624a91950d8 service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:53.796939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1060/2126] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:10:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 21 15:10:53.803639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1067/2127] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:53 2022] DELETE /identity/v3/users/08ed74ad292841aab9d36d54e5a189c7 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:53.809950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:53.810043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:53.810338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:53.830484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:53.831736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:53.878160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:53.892869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1061/2128] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:53.906471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:53.907921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:53.971119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c34015-84b4-4539-a49f-e9f20c4f53e9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:53.975145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:53.982240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:53.982240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:53.982240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:53.992796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:53.994983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:54.011844 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1068/2129] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:54.681414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:54.683003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:54.710478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90cd0b3-c96d-4241-b603-a0cbe6c65a2a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:54.721330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:54.722185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:54.722621 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:54.723095 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] PATH_INFO: `/v3/projects/9fd1bdf287b3429b9c1a69034b74d149` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:54.724458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd1bdf287b3429b9c1a69034b74d149)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:54.727897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:54.863039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd1bdf287b3429b9c1a69034b74d149', 'request_id': 'req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:54.934109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd1bdf287b3429b9c1a69034b74d149', 'request_id': 'req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:54.940745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd1bdf287b3429b9c1a69034b74d149', 'request_id': 'req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:54.963108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] The token cache is being invalidate because project 9fd1bdf287b3429b9c1a69034b74d149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:54.963327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:54.967616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:54.972319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1062/2130] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:54 2022] DELETE /identity/v3/projects/9fd1bdf287b3429b9c1a69034b74d149 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:54.973743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:54.988837 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483fad-4f29-40c1-b671-c5f2d505cad0 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:55.015886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:55.064412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:55.065391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:55.065858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:55.066126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] PATH_INFO: `/v3/users/d325a72c97c545b582a21c94c8ec7402` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:55.067413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d325a72c97c545b582a21c94c8ec7402)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:55.069434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:55.252775 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd325a72c97c545b582a21c94c8ec7402'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:55.279622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd325a72c97c545b582a21c94c8ec7402'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:55.280376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd325a72c97c545b582a21c94c8ec7402'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:55.293412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd325a72c97c545b582a21c94c8ec7402'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:55.296454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f05cac58-4b51-43bc-800f-f86c918f9c85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd325a72c97c545b582a21c94c8ec7402'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:55.377277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1069/2131] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:54 2022] DELETE /identity/v3/users/d325a72c97c545b582a21c94c8ec7402 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:10:55.995609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:55.997051 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:56.022935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a15634-1fc3-4e3c-b664-3e5e5f3ac50d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.092683 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.093429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.093775 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.094115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] PATH_INFO: `/v3/projects/5acd30143242435aa70986d0d9e0d4ad` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.095351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acd30143242435aa70986d0d9e0d4ad)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.097990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.219327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acd30143242435aa70986d0d9e0d4ad', 'request_id': 'req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:56.301537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acd30143242435aa70986d0d9e0d4ad', 'request_id': 'req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:56.308718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acd30143242435aa70986d0d9e0d4ad', 'request_id': 'req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:56.322726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] The token cache is being invalidate because project 5acd30143242435aa70986d0d9e0d4ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:56.325654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:56.325654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-99d5a63c-3d5a-4767-ab25-b4c5fd82c36a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:56.326374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1063/2132] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:55 2022] DELETE /identity/v3/projects/5acd30143242435aa70986d0d9e0d4ad => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:56.333132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.333729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.333729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.362584 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:56.363851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:56.431693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1070/2133] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:56.499297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.499586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.499870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.511275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:56.531809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:56.533488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:56.549987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1064/2134] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:10:56.551386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:56.564542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22312abc-f6de-45ac-b64c-bf296c2cbec6 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.571054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.572080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.572573 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.573083 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.573759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:56.574329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.576672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.595164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1071/2135] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:56.596122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:56.607629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f55be3-028e-4566-8a46-a8c593997f65 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.612673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.613300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.613582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.613857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.614536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.616489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.704391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1065/2136] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:56 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:56.709620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:56.721902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe38ad2-9fa2-4de2-b9af-4a35e5ae3548 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.753924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:56.790021 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.790656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.790944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.791217 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.791853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.793866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.879474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1072/2137] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:56 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:56.884880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:56.896616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4690bb-2a8b-4082-8119-ce8dab69f32b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.902086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.902725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.903004 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.903273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.904052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.905562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.918344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1066/2138] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:10:56.923009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:56.946360 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c500fdc2-fc6f-4cca-b6ef-b5f977c47a91 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:56.964843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:56.965479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:56.965781 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:56.966026 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:56.966718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:56.968332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9bc168-74b9-43f6-9edd-e382658e99d5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:56.991199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1073/2139] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:56.993563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:57.010500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db2db1-ebce-4a66-9ddd-337589ea2da0 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:57.018601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:57.019711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:57.020099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:57.020637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] PATH_INFO: `/v3/projects/e8cfccb2a72d451598599b80d37c438a/users/cbb20382a77a471e8496c25da7047581/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:57.054208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8cfccb2a72d451598599b80d37c438a, user_id=cbb20382a77a471e8496c25da7047581, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:57.057922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:57.169455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1067/2140] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:56 2022] PUT /identity/v3/projects/e8cfccb2a72d451598599b80d37c438a/users/cbb20382a77a471e8496c25da7047581/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:57.182180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:57.182560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:57.182747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:57.214215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] MFA Rules not processed for user `cbb20382a77a471e8496c25da7047581`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:57.215472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:57.300510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1074/2141] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:57.313622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:57.315355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:57.378834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cedb0c-6371-4101-8ee5-84228eddb931 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:57.382888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:57.383583 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:57.383928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:57.384269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:57.393728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:57.395327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6504e355-b0f7-4df7-93a3-799b182fae68 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:57.407345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1068/2142] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:57.438085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:57.507761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615436f4-30f3-4df9-ae4a-2e687c13b247 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:57.512863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:57.513751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:57.514136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:57.514388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] PATH_INFO: `/v3/users/cbb20382a77a471e8496c25da7047581` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:57.515672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbb20382a77a471e8496c25da7047581)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:57.518189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:57.785512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:57.792632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbb20382a77a471e8496c25da7047581'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:57.808193 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbb20382a77a471e8496c25da7047581'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:57.809297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbb20382a77a471e8496c25da7047581'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:57.818539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbb20382a77a471e8496c25da7047581'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:57.819448 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbb20382a77a471e8496c25da7047581'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:57.907585 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1075/2143] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:10:57 2022] DELETE /identity/v3/users/cbb20382a77a471e8496c25da7047581 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:57.917786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:57.918342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:57.918612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:57.938602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:57.939904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:58.014678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1069/2144] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:58.025159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:58.026588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:58.098864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961fa6ea-ed23-4b4b-a7a9-d8bc900e5254 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:58.102647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:58.103267 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:58.103582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:58.103845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:58.113964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:58.115536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a678d4-525b-400f-8d4c-faaf93ce7017 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:58.128472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1076/2145] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:10:58.669624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:58.671587 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:58.687228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68dba4-a232-4193-91bf-534269d28c4b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:58.693120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:58.694041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:58.694784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:58.696676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] PATH_INFO: `/v3/projects/e8cfccb2a72d451598599b80d37c438a` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:58.698830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8cfccb2a72d451598599b80d37c438a)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:58.700888 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:58.802334 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:58.819647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8cfccb2a72d451598599b80d37c438a', 'request_id': 'req-132adaf4-f74e-4a11-a5c5-d28284c4c57e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:58.868211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8cfccb2a72d451598599b80d37c438a', 'request_id': 'req-132adaf4-f74e-4a11-a5c5-d28284c4c57e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:58.874860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8cfccb2a72d451598599b80d37c438a', 'request_id': 'req-132adaf4-f74e-4a11-a5c5-d28284c4c57e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:58.889566 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] The token cache is being invalidate because project e8cfccb2a72d451598599b80d37c438a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:10:58.889857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:58.890520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-132adaf4-f74e-4a11-a5c5-d28284c4c57e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:10:58.892098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1070/2146] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:10:58 2022] DELETE /identity/v3/projects/e8cfccb2a72d451598599b80d37c438a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:58.897014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:58.897521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:58.897627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:58.924660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:58.925848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:59.009016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1077/2147] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:59.016798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.017322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.017411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.051067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:59.053325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:59.080190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1071/2148] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:59.084534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.096077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c9983-18cb-46d9-b1fb-aca65fa5a5b7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.102221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.102952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.108351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.108755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.109642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:10:59.109956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.111627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.131148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.146196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1078/2149] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:10:59.149353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c681aa1-52ec-43f9-b63a-9f71d0209d4b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.154297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.155090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.155373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.155797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.156449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.158380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.233830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1072/2150] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:10:59 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:10:59.235229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.246428 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576b188-8f0f-4764-abbd-d78713bf929c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.300351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:10:59.315524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.322121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.322121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.322121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.322121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.322121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.420793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1079/2151] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:10:59 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:10:59.420952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.443639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303cb419-6e9d-48fb-9170-09b596d9ba6a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.449363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.449981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.460790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.460790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.460790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.460790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.463738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1073/2152] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:59.467508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.482952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b62e7-da7a-4239-bd42-ac25ed54f423 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.488550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.489255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.489610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.489970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.490736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.493024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fd67a4-9a5b-40c1-8057-68ccc0816c03 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.505998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1080/2153] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:10:59.511593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.524194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ff606a-c806-41c2-9135-1eabfd88eeb7 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.529341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.530141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.530528 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.530938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] PATH_INFO: `/v3/projects/afe7e2e48b37425f8bf533fbb3ca634d/users/7fc6194063a3415e80b763e521b8a344/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.548594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afe7e2e48b37425f8bf533fbb3ca634d, user_id=7fc6194063a3415e80b763e521b8a344, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.551652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.648244 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1074/2154] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:10:59 2022] PUT /identity/v3/projects/afe7e2e48b37425f8bf533fbb3ca634d/users/7fc6194063a3415e80b763e521b8a344/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:10:59.660917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11da42-20e9-4e5a-b7de-52bfd8be39e8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.661220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11da42-20e9-4e5a-b7de-52bfd8be39e8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.661462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11da42-20e9-4e5a-b7de-52bfd8be39e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.694499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4e11da42-20e9-4e5a-b7de-52bfd8be39e8 None None] MFA Rules not processed for user `7fc6194063a3415e80b763e521b8a344`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:10:59.695942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4e11da42-20e9-4e5a-b7de-52bfd8be39e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:59.780804 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1081/2155] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:10:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:10:59.794023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:10:59.796479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:10:59.870824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5687584-4e0c-4944-bcb7-ee12ebe1b557 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:10:59.874762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:10:59.875455 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:10:59.875736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:10:59.876130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:10:59.888299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:10:59.889866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:10:59.899860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1075/2156] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:10:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:09.776683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:09.895140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:11.524250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a85a0-457e-4077-b279-733835a997cb None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:11.524988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a85a0-457e-4077-b279-733835a997cb None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:11.524988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a85a0-457e-4077-b279-733835a997cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:11.547948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e39a85a0-457e-4077-b279-733835a997cb None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:11.549199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e39a85a0-457e-4077-b279-733835a997cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:11.647328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1082/2157] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:11:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:11.665007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:11.666507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:11.743561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3242fd-6fb5-40aa-990c-6bad0b778c05 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:11.750751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:11.751604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:11.752802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:11.753074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:11.765082 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:11.769150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d542ce6-856f-4a84-a288-cd8d1955bb31 service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:11.776176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1076/2158] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:11:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:11:12.194352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a85a0-457e-4077-b279-733835a997cb None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:12.275291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a85a0-457e-4077-b279-733835a997cb None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:12.282283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:12.282968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:12.283250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:12.283525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] PATH_INFO: `/v3/users/7fc6194063a3415e80b763e521b8a344` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:12.285117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc6194063a3415e80b763e521b8a344)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:12.287521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:12.492784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc6194063a3415e80b763e521b8a344'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:12.505211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc6194063a3415e80b763e521b8a344'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:12.505866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc6194063a3415e80b763e521b8a344'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:12.513062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc6194063a3415e80b763e521b8a344'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:12.513722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc6194063a3415e80b763e521b8a344'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:12.672656 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1083/2159] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:12 2022] DELETE /identity/v3/users/7fc6194063a3415e80b763e521b8a344 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:12.676868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:12.677308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:12.677428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:12.697379 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:12.698620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:12.764234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1077/2160] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:11:12.775037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:12.776521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:12.826819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:12.847680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229d28a-014d-4771-812e-80f031342df9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:12.851412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:12.852802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:12.852910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:12.853210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:12.862283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:12.863737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:12.876206 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1084/2161] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:13.361529 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:13.363401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:13.377036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a30903-495e-44f0-ae25-05aed2f02b89 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:13.381776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:13.382377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:13.382686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:13.383556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] PATH_INFO: `/v3/projects/afe7e2e48b37425f8bf533fbb3ca634d` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:13.384931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afe7e2e48b37425f8bf533fbb3ca634d)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:13.386952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:13.541799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afe7e2e48b37425f8bf533fbb3ca634d', 'request_id': 'req-8d973a77-e44f-49c7-aa92-c553c5649ff0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:13.614324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afe7e2e48b37425f8bf533fbb3ca634d', 'request_id': 'req-8d973a77-e44f-49c7-aa92-c553c5649ff0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:13.621876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afe7e2e48b37425f8bf533fbb3ca634d', 'request_id': 'req-8d973a77-e44f-49c7-aa92-c553c5649ff0'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:13.634091 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] The token cache is being invalidate because project afe7e2e48b37425f8bf533fbb3ca634d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:11:13.634344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:13.634980 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:13.637289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1078/2162] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:11:13 2022] DELETE /identity/v3/projects/afe7e2e48b37425f8bf533fbb3ca634d => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:14.310861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:14.312789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:14.402911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4bdb2a-1221-4899-89e4-a53cdd559969 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:14.479985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:14.480625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:14.480907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:14.481180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:14.483015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:14.547745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:14.549588 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:14.562831 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1085/2163] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:19.441331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:19.443394 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:19.457638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:19.500715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d973a77-e44f-49c7-aa92-c553c5649ff0 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:19.504367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:19.504964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:19.505243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:19.505514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:19.507283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:19.569315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:19.570780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:19.580356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1079/2164] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:23.266582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:23.268105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:23.273344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:23.348229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9aa4de-778a-4656-9172-25441e834ff7 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:23.353285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:23.353886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:23.354160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:23.354429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:23.357184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:23.368346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:23.373921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:23.380968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1086/2165] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:23.483598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:23.485672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:23.508695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c29bbe7-e525-47fd-a291-8e28a9c6b2e2 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:23.514510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:23.516261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:23.516261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:23.516261 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] PATH_INFO: `/v3/users/1eb3641974744d83816b6f3ad7bb6f82` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:23.517092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eb3641974744d83816b6f3ad7bb6f82)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:23.519289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:23.760997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eb3641974744d83816b6f3ad7bb6f82'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:23.761704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eb3641974744d83816b6f3ad7bb6f82'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:23.773993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eb3641974744d83816b6f3ad7bb6f82'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:23.867262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eb3641974744d83816b6f3ad7bb6f82'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:23.867817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-78d318c6-3d82-4be9-b7ac-4707b9a7b3af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eb3641974744d83816b6f3ad7bb6f82'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:23.880628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1080/2166] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:23 2022] DELETE /identity/v3/users/1eb3641974744d83816b6f3ad7bb6f82 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:24.211191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:24.224795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f9c83-6567-4b7b-88fc-d84aec9b8d08 service placement] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:24.283719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:24.284382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:24.284619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:24.284895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] PATH_INFO: `/v3/projects/77156a46a45648cd8cb0ea22ab0f04a8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:24.286022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77156a46a45648cd8cb0ea22ab0f04a8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:24.287998 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:24.439856 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77156a46a45648cd8cb0ea22ab0f04a8', 'request_id': 'req-a637105c-a696-4e07-907c-195dbaa9be70'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:24.615890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77156a46a45648cd8cb0ea22ab0f04a8', 'request_id': 'req-a637105c-a696-4e07-907c-195dbaa9be70'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:24.623603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77156a46a45648cd8cb0ea22ab0f04a8', 'request_id': 'req-a637105c-a696-4e07-907c-195dbaa9be70'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:24.634190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] The token cache is being invalidate because project 77156a46a45648cd8cb0ea22ab0f04a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:11:24.634481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:24.635159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a637105c-a696-4e07-907c-195dbaa9be70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:24.636722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1087/2167] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:11:24 2022] DELETE /identity/v3/projects/77156a46a45648cd8cb0ea22ab0f04a8 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:24.642227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:24.642726 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:24.643107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:24.663126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:24.664405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:24.697465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:24.729618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1081/2168] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:24.737226 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:24.737511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:24.737786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:24.760434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:24.762315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:24.771643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1088/2169] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:24.776068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:24.785989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1227235-e22a-424c-8c01-264b4538cde4 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:24.794057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:24.794697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:24.794978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:24.795247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:24.796155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:11:24.796457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:24.798081 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:24.810938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1082/2170] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:11:24.815873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:24.825154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3849fdf-bf96-4fee-8e86-7b792b193e0a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:24.829995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:24.830700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:24.831046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:24.831384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:24.832093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:24.838548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:24.849028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:24.988763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1089/2171] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:24 2022] POST /identity/v3/projects => generated 355 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:24.993079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.004115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cedf2c6-b73d-4ea1-afce-a7e8441123d8 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.060123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.065931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.065931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.065931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.065931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.065931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.188415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1083/2172] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:24 2022] POST /identity/v3/users => generated 278 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:25.192500 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.202920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d477d0a-294c-459e-af79-3838616e07a3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.207437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.208023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.208295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.208561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.209232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.210835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.221361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1090/2173] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:25.225304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.236212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfabe8c-bc7b-4234-b080-ab2f501a2de0 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.240852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.241441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.241720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.241991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.242641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.244292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455ff337-3b25-463e-abb3-a4c2235d20d8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.255485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1084/2174] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:25.259744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.270230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a44f4-e665-4304-8f6d-24db5916cf63 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.274881 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.275530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.275803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.276075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] PATH_INFO: `/v3/projects/e7213927abb74ce4a02a9524fb1cb5b1/users/0c5e2c53eff44344a02656b73c90357c/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.291777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7213927abb74ce4a02a9524fb1cb5b1, user_id=0c5e2c53eff44344a02656b73c90357c, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.295396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.427419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1091/2175] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:25 2022] PUT /identity/v3/projects/e7213927abb74ce4a02a9524fb1cb5b1/users/0c5e2c53eff44344a02656b73c90357c/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:25.439034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.439340 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.439617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.468741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] MFA Rules not processed for user `0c5e2c53eff44344a02656b73c90357c`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:25.469983 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:25.533972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1085/2176] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:25.538534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.616766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15c5ab4-e3af-4fd3-9dbf-9ad89ae477c1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.621851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.622487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.622812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.623086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.623725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.625736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.757295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1092/2177] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:25 2022] POST /identity/v3/projects => generated 355 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:25.758516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.771031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09c8585-141a-4efc-9d0c-83d52467ccd1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.800070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:25.826947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.827551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.827827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.828092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.828714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.830615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:25.970738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1086/2178] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:25 2022] POST /identity/v3/users => generated 284 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:25.975065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:25.985617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca81d1a8-977f-4c58-b3f9-6bc82ba197d0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:25.990223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:25.990857 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:25.991136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:25.991406 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:25.992040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:25.993625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.005827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1093/2179] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:26.010028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:26.021111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc047b-27b4-4831-90da-0adccd90f26b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:26.025707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:26.026374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.026674 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.026950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] PATH_INFO: `/v3/projects/3931968cf1874aac95e84d6de915ecc7/users/cee30ee6c7f74a1fabbe84c6da509c5e/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.042041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3931968cf1874aac95e84d6de915ecc7, user_id=cee30ee6c7f74a1fabbe84c6da509c5e, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:26.045287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.158344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1087/2180] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:26 2022] PUT /identity/v3/projects/3931968cf1874aac95e84d6de915ecc7/users/cee30ee6c7f74a1fabbe84c6da509c5e/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:26.162308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:26.172771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea5357-b7fc-45cc-9ae0-e55efe0a8b11 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:26.229416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:26.230014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.230290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.230599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.231308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:26.232882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9c2516-d6dd-46a8-a3db-33816a9b8df2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.242444 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1094/2181] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:26.247224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:26.257315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed558316-7c4b-4f0a-8efb-b3112885db95 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:26.262489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:26.263156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.263432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.263704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] PATH_INFO: `/v3/projects/3931968cf1874aac95e84d6de915ecc7/users/cee30ee6c7f74a1fabbe84c6da509c5e/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.265609 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3931968cf1874aac95e84d6de915ecc7, user_id=cee30ee6c7f74a1fabbe84c6da509c5e, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:26.268273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.278570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1088/2182] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:26 2022] PUT /identity/v3/projects/3931968cf1874aac95e84d6de915ecc7/users/cee30ee6c7f74a1fabbe84c6da509c5e/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:26.288740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.289024 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.289299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.319651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] MFA Rules not processed for user `cee30ee6c7f74a1fabbe84c6da509c5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:26.320935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:26.338369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:26.389888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1095/2183] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:26 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:26.401892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:26.404166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:26.463359 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3acb54-4cc4-4cd5-8616-8a00e11ec86e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:26.467584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:26.468179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.468456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.468727 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.477754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:26.479237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.489408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1089/2184] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:26.760994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:26.770555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d71f8-1b43-42f4-880d-b83079c7a326 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:26.774377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:26.775059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:26.775342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:26.775616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:26.837285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:26.838818 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054741bb-0a34-4bbe-acd9-a5058875e8eb service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:26.848669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1096/2185] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:36.485127 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:36.844374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=49 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:44.819321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:44.888637 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547788a3-6770-4a52-8349-802ce8b3b756 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:44.893911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:44.895177 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:44.895456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:44.895729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] PATH_INFO: `/v3/users/0c5e2c53eff44344a02656b73c90357c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:44.897012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c5e2c53eff44344a02656b73c90357c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:44.899083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:45.095030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c5e2c53eff44344a02656b73c90357c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:45.095708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c5e2c53eff44344a02656b73c90357c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:45.108483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c5e2c53eff44344a02656b73c90357c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:45.186557 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c5e2c53eff44344a02656b73c90357c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:45.187199 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c5e2c53eff44344a02656b73c90357c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:45.200462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1090/2186] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:44 2022] DELETE /identity/v3/users/0c5e2c53eff44344a02656b73c90357c => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:45.205049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:45.205527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:45.205657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:45.225753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:45.227000 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:45.288566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1097/2187] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:45.300030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:45.300405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:45.357940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd45d88-6983-49c1-89fb-057796e944da admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:45.361627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:45.362818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:45.363100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:45.363369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:45.372290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:45.373873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:45.383898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1091/2188] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:45.775660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:45.777250 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:45.792354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90ec6d-199f-47c7-b5a8-a9e06f2f1eb4 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:45.797441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:45.798125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:45.798404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:45.798733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] PATH_INFO: `/v3/projects/e7213927abb74ce4a02a9524fb1cb5b1` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:45.800524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7213927abb74ce4a02a9524fb1cb5b1)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:45.802605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:45.924313 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7213927abb74ce4a02a9524fb1cb5b1', 'request_id': 'req-ea3c66ee-324f-4400-bb47-1b4547de64a7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.004330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7213927abb74ce4a02a9524fb1cb5b1', 'request_id': 'req-ea3c66ee-324f-4400-bb47-1b4547de64a7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.011298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7213927abb74ce4a02a9524fb1cb5b1', 'request_id': 'req-ea3c66ee-324f-4400-bb47-1b4547de64a7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.022035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] The token cache is being invalidate because project e7213927abb74ce4a02a9524fb1cb5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:11:46.022322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.022972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.024463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1098/2189] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:11:45 2022] DELETE /identity/v3/projects/e7213927abb74ce4a02a9524fb1cb5b1 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:46.028552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:46.039346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd8ea2-d310-4622-a4cb-4f4eb11163a7 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:46.058514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:46.099153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:46.099794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:46.100109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:46.100367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] PATH_INFO: `/v3/users/cee30ee6c7f74a1fabbe84c6da509c5e` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:46.101511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cee30ee6c7f74a1fabbe84c6da509c5e)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:46.103538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:46.234146 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cee30ee6c7f74a1fabbe84c6da509c5e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.234831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cee30ee6c7f74a1fabbe84c6da509c5e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.247016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cee30ee6c7f74a1fabbe84c6da509c5e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.335080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cee30ee6c7f74a1fabbe84c6da509c5e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.335717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-3f39b317-ec26-4725-b64d-df06dd2ee42b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cee30ee6c7f74a1fabbe84c6da509c5e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:46.351243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1092/2190] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:46 2022] DELETE /identity/v3/users/cee30ee6c7f74a1fabbe84c6da509c5e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:46.742519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:46.743985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:46.762684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c66ee-324f-4400-bb47-1b4547de64a7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:46.823468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:46.824103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:46.824372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:46.824726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] PATH_INFO: `/v3/projects/3931968cf1874aac95e84d6de915ecc7` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:46.825747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3931968cf1874aac95e84d6de915ecc7)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:46.827892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:46.942792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:46.958455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3931968cf1874aac95e84d6de915ecc7', 'request_id': 'req-dde03ce4-f1a3-41fa-bae3-0104f634ee11'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:47.026920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3931968cf1874aac95e84d6de915ecc7', 'request_id': 'req-dde03ce4-f1a3-41fa-bae3-0104f634ee11'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:47.033933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3931968cf1874aac95e84d6de915ecc7', 'request_id': 'req-dde03ce4-f1a3-41fa-bae3-0104f634ee11'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:47.044244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] The token cache is being invalidate because project 3931968cf1874aac95e84d6de915ecc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:11:47.044532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:47.045157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dde03ce4-f1a3-41fa-bae3-0104f634ee11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:47.046934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1099/2191] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:11:46 2022] DELETE /identity/v3/projects/3931968cf1874aac95e84d6de915ecc7 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:47.053598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.054128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.054251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.074605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:47.075944 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:47.135753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1093/2192] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:11:47.145142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.145667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.146187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.166931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:47.168148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:47.177154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1100/2193] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:47.181329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.191053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a117d7f-87c5-4e30-96b9-91e6063fef71 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.195947 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.196413 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.196694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.196967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.197612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:11:47.197915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.199518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.214315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1094/2194] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:11:47.215415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.226369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889adc48-7a55-4167-8d1e-70ca95d8f777 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.231157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.231760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.232041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.232310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.232946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.235012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.324518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1101/2195] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/projects => generated 407 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:11:47.328536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.339077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9177d-dc6e-46e6-8f6a-0a1919c5b949 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.367750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:47.397826 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.398458 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.398763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.399064 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.399682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.401715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.507310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1095/2196] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/users => generated 304 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:47.511301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.522166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4c60a-41ed-475f-aab9-b825ed048834 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.527157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.527820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.528127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.528420 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.529046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.530664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.541388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1102/2197] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:47.545664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.556770 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9243ff2-62c7-42ec-b9c4-036081cc8440 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.561549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.562172 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.562454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.562761 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.563396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.564981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f396f-303f-4b67-87c6-03bb1f33490a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.575628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1096/2198] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:47.579766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.590452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e650df-941d-47de-b86c-7d7d6f7b1cb6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.595332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.596115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.596263 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.596524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] PATH_INFO: `/v3/projects/a9d2eb083d274aaab445df597b13cbf8/users/214b719eea7f4d42b8615050297ee3e1/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.612902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9d2eb083d274aaab445df597b13cbf8, user_id=214b719eea7f4d42b8615050297ee3e1, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.615665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.718902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1103/2199] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:47 2022] PUT /identity/v3/projects/a9d2eb083d274aaab445df597b13cbf8/users/214b719eea7f4d42b8615050297ee3e1/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:11:47.729791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.730080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.730358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.760892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] MFA Rules not processed for user `214b719eea7f4d42b8615050297ee3e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:47.762111 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:47.834501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1097/2200] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:47.839015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:47.901855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b409e-db6f-4ea0-94e2-045ceb192f71 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:47.911256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:47.911927 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:47.912203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:47.912471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:47.913114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:47.915153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:47.999320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1104/2201] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:47 2022] POST /identity/v3/projects => generated 407 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:11:48.003509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.014803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f380da6-5689-4990-a226-c3bed4e3bd7b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.071604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.072235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.072391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.072665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.073290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.075280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.158158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1098/2202] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:48 2022] POST /identity/v3/users => generated 310 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:48.162877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.173256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fdf8b1-7a2b-4909-861a-44888ced0441 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.178018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.178612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.178918 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.179239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.179891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.181478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.193668 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1105/2203] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:48.197811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.208268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6e667-7475-4961-a5c3-423c357a4604 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.213028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.213737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.214018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.214288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] PATH_INFO: `/v3/projects/80c0266ae9684c838b98a30035e256cd/users/35beda1a3b9144e4b52654f5c7b61219/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.228843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c0266ae9684c838b98a30035e256cd, user_id=35beda1a3b9144e4b52654f5c7b61219, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.231511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.310379 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1099/2204] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:48 2022] PUT /identity/v3/projects/80c0266ae9684c838b98a30035e256cd/users/35beda1a3b9144e4b52654f5c7b61219/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:11:48.314445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.325411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac0de4-f9cd-4d90-a32a-f0b97d566c9f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.345196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:48.384518 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.385126 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.385474 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.385761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.386410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.388037 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60186806-9be7-4ce5-a625-d104998aec7b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.398195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1106/2205] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:48.402391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.407880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:48.413238 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bb44bf-3bc2-4abd-8ad5-c11379f2b3e5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.417841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.418475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.418781 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.419055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] PATH_INFO: `/v3/projects/80c0266ae9684c838b98a30035e256cd/users/35beda1a3b9144e4b52654f5c7b61219/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.421219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80c0266ae9684c838b98a30035e256cd, user_id=35beda1a3b9144e4b52654f5c7b61219, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.424040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.434543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1100/2206] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:48 2022] PUT /identity/v3/projects/80c0266ae9684c838b98a30035e256cd/users/35beda1a3b9144e4b52654f5c7b61219/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:48.445381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.445664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.445938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.474405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] MFA Rules not processed for user `35beda1a3b9144e4b52654f5c7b61219`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:48.475673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:48.543023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1107/2207] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:48.554697 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:48.556424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:48.613743 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefff2c2-835c-4f8c-9434-99b4c517f35c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:48.617435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:48.618032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:48.618310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:48.618581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:48.627632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:48.629811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:48.639224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1101/2208] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:49.036295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:49.051934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ee973-0a72-46c8-b08a-b62ca7dbe3b8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:49.058433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:49.059408 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:49.060138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:49.060601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:49.096753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:49.098851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:49.135049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:49.136416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:49.149172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1108/2209] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:49 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:49.167982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ea822f-212c-4ffa-b032-29c8bac51cb6 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:49.171981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:49.172589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:49.173641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:49.173641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:49.175310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:49.250074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:49.251658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:49.262677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1102/2210] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:11:53.114873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:53.116867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:53.164164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:53.187295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701db9db-45d0-4442-b36a-914681fad2e9 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:53.192371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:53.192979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:53.193261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:53.193612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:53.195339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:53.207032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:53.208643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:53.221685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1109/2211] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:53.221861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:53.223339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:53.240292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7f47d-6c0f-4f80-bde8-c6de3619524f service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:53.248337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:53.248337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:53.248337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:53.248337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] PATH_INFO: `/v3/users/3418ee58bd51418d9594b98ed594f506` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:53.248877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3418ee58bd51418d9594b98ed594f506)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:53.251266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:53.420358 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:53.429723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3418ee58bd51418d9594b98ed594f506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:53.430401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3418ee58bd51418d9594b98ed594f506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:53.442204 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3418ee58bd51418d9594b98ed594f506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:53.584471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3418ee58bd51418d9594b98ed594f506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:53.584471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a5a3978d-ef73-4fa3-98ac-20302c508b79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3418ee58bd51418d9594b98ed594f506'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:53.596151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1103/2212] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:53 2022] DELETE /identity/v3/users/3418ee58bd51418d9594b98ed594f506 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:11:53.878993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:53.893148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328a07df-7171-493e-9d0e-a4d5ea2f8684 service placement] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:53.954417 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:53.954963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:53.955255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:53.955499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] PATH_INFO: `/v3/projects/0025ac2cdc0f45dab742a02486307716` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:53.956829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0025ac2cdc0f45dab742a02486307716)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:53.958779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.106851 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0025ac2cdc0f45dab742a02486307716', 'request_id': 'req-5d178599-c9be-416b-9673-40585ab0b26f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:54.197680 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0025ac2cdc0f45dab742a02486307716', 'request_id': 'req-5d178599-c9be-416b-9673-40585ab0b26f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:54.205295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0025ac2cdc0f45dab742a02486307716', 'request_id': 'req-5d178599-c9be-416b-9673-40585ab0b26f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:54.216412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] The token cache is being invalidate because project 0025ac2cdc0f45dab742a02486307716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:11:54.216700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:54.217338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5d178599-c9be-416b-9673-40585ab0b26f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:54.218954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1110/2213] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:11:53 2022] DELETE /identity/v3/projects/0025ac2cdc0f45dab742a02486307716 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:54.223848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.224344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.224589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.244544 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:54.245741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:54.303945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1104/2214] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:54.311947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.312231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.312505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.332637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:54.334194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:54.341913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1111/2215] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:54.346135 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.356021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff80377-f2b0-4eb0-878e-6afc17370ef7 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.360656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.361257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.361538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.361813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.362491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:11:54.362823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.364409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.376577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1105/2216] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:11:54.380860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.390535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54d94a-706f-4861-a7c8-4916e866629f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.395164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.395749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.396023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.396286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.397156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.399096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.501465 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1112/2217] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:54 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:11:54.505400 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.516299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265205e5-87f2-4fe5-bec8-eb56bc85c396 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.576429 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.577035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.577312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.577579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.578284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.580252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.712342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1106/2218] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:54 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:54.716580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.727414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69759239-e890-4afa-b92e-ac07404606d2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.732212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.732806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.733087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.733382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.734027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.735643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.746179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1113/2219] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:54.750193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.760718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d0652-74e5-4768-a62a-92ba358a804a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.765191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.765778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.766054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.766324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.769065 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.770718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a393c83-2552-4516-9357-aef70da5592d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.784070 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1107/2220] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:54.791106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:54.801812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197483de-013b-45b4-af9e-4261b774aa60 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:54.813213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:54.813947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.814227 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.814614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] PATH_INFO: `/v3/projects/73f58dbdf1c745ea81b943c73a400955/users/6b03390675a04b03aeff99282bfd2e02/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.830264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f58dbdf1c745ea81b943c73a400955, user_id=6b03390675a04b03aeff99282bfd2e02, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:54.833007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:54.917953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:54.922799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1114/2221] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:54 2022] PUT /identity/v3/projects/73f58dbdf1c745ea81b943c73a400955/users/6b03390675a04b03aeff99282bfd2e02/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:54.934336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:54.934712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:54.934934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:54.964645 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] MFA Rules not processed for user `6b03390675a04b03aeff99282bfd2e02`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:54.965869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:54.974245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:55.033375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1108/2222] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:55.038519 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.100288 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b743ccf-329d-412d-a979-225d40051d33 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.104814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.110601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.110601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.110601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.110601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.110601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.193424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1115/2223] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:11:55 2022] POST /identity/v3/projects => generated 381 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:55.197768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.208569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836d11c4-ff79-4ef4-95ab-ca24089befc0 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.264982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.265704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.265934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.266203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.266857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.268745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.369320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1109/2224] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:11:55 2022] POST /identity/v3/users => generated 297 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:11:55.373258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.384885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428b202-8ae7-48a1-801d-6b5e407c2695 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.389639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.390046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.390324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.390593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.391293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.392883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.403962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1116/2225] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:55.408063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.418151 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271e9d3-72d7-4b48-9880-c53a97b18f15 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.423220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.423862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.424149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.424420 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] PATH_INFO: `/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08/users/98275bd2a53048628bf1ab808c7fc3f6/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.440161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0ddf09a05e4c96b784ad7215fa1d08, user_id=98275bd2a53048628bf1ab808c7fc3f6, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.442856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.529857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1110/2226] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:55 2022] PUT /identity/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08/users/98275bd2a53048628bf1ab808c7fc3f6/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:55.533935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.544594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3935f5-b570-4794-ba56-a0d93ace7bbd admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.602447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.603085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.603361 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.603630 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.604266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.605846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f0927-e0e2-41d8-8111-aef075e31763 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.616131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1117/2227] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:11:55.624882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.631207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721e0c8-8998-4071-834b-50015d3b3aeb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.636301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.636932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.637208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.637481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] PATH_INFO: `/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08/users/98275bd2a53048628bf1ab808c7fc3f6/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.639524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0ddf09a05e4c96b784ad7215fa1d08, user_id=98275bd2a53048628bf1ab808c7fc3f6, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.642184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.653366 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1111/2228] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:11:55 2022] PUT /identity/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08/users/98275bd2a53048628bf1ab808c7fc3f6/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:55.665399 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.665689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.665968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.695477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] MFA Rules not processed for user `98275bd2a53048628bf1ab808c7fc3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:55.696751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:55.758084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:55.767070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1118/2229] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:55.779407 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:55.781273 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:55.839985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08b7d8-a15b-497d-be76-d1255ef06cb5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:55.843618 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:55.844219 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:55.844521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:55.844791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:55.853905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:55.855409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:55.865816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1112/2230] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:56.253221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:56.257514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15190d5-ce4a-45cf-9d87-7fe20f9b238c None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:56.261386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:56.261990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:56.262264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:56.262531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:56.336215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:56.338039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:56.348356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1119/2231] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:11:57.400211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:57.401599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:57.408044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:11:57.458196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd710fa-25cf-4eed-811a-4070257ea3e4 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:57.463024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:57.463024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:57.463231 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:57.463419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:57.465276 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:57.529522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:57.531002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a0a70-d06b-49a1-bfb0-808e4df73a67 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:57.541208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1113/2232] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:11:59.394152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:59.395841 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:59.413373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cdd11-0ec1-44c5-9bee-c112ed2babff service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:11:59.419011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:11:59.419862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:59.420419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:59.420739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] PATH_INFO: `/v3/users/214b719eea7f4d42b8615050297ee3e1` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:59.422311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=214b719eea7f4d42b8615050297ee3e1)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:11:59.424405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:11:59.688008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214b719eea7f4d42b8615050297ee3e1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:59.704578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214b719eea7f4d42b8615050297ee3e1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:59.706260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214b719eea7f4d42b8615050297ee3e1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:59.721892 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214b719eea7f4d42b8615050297ee3e1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:59.722752 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214b719eea7f4d42b8615050297ee3e1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:11:59.808413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1120/2233] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:11:59 2022] DELETE /identity/v3/users/214b719eea7f4d42b8615050297ee3e1 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:11:59.812800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:11:59.813338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:11:59.813338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:11:59.835205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:11:59.836734 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:11:59.931740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1114/2234] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:11:59.932845 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:11:59.934485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:00.031174 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e978818-add5-4db5-b6dc-9457b67b5eec admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:00.034761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:00.035357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:00.035631 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:00.035893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:00.046056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:00.047879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:00.066883 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1121/2235] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:00.570553 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:00.571998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:00.584530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2a9aa-5b6e-4193-9d33-706deb314110 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:00.589600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:00.590242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:00.590517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:00.590814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] PATH_INFO: `/v3/projects/a9d2eb083d274aaab445df597b13cbf8` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:00.591966 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9d2eb083d274aaab445df597b13cbf8)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:00.593877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:00.697196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9d2eb083d274aaab445df597b13cbf8', 'request_id': 'req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:00.698902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:00.797270 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9d2eb083d274aaab445df597b13cbf8', 'request_id': 'req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:00.806593 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9d2eb083d274aaab445df597b13cbf8', 'request_id': 'req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:00.818317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] The token cache is being invalidate because project a9d2eb083d274aaab445df597b13cbf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:00.819038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:00.819614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:00.821975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1115/2236] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:00 2022] DELETE /identity/v3/projects/a9d2eb083d274aaab445df597b13cbf8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:00.825977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:00.835391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a2d3c-d4ea-4767-a476-942e7cdcabd4 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:00.840783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:00.895100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:00.895736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:00.896011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:00.896280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] PATH_INFO: `/v3/users/35beda1a3b9144e4b52654f5c7b61219` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:00.897409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35beda1a3b9144e4b52654f5c7b61219)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:00.899678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:01.123455 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35beda1a3b9144e4b52654f5c7b61219'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.146204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35beda1a3b9144e4b52654f5c7b61219'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.147085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35beda1a3b9144e4b52654f5c7b61219'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.155038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35beda1a3b9144e4b52654f5c7b61219'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.155895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35beda1a3b9144e4b52654f5c7b61219'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.238172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1122/2237] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:00 2022] DELETE /identity/v3/users/35beda1a3b9144e4b52654f5c7b61219 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:01.730608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:01.735846 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:01.738625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:01.740234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:01.755975 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6ecd7-3b89-45ba-8a8b-585f47143fd5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:01.814080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc4c6c0-f1c9-483d-99a3-ebe699648c49 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:01.818797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:01.819434 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:01.819712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:01.819983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] PATH_INFO: `/v3/projects/80c0266ae9684c838b98a30035e256cd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:01.821160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] RBAC: Authorizing `identity:delete_project(project_id=80c0266ae9684c838b98a30035e256cd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:01.823100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:01.824530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:01.825196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:01.825657 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:01.825795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] PATH_INFO: `/v3/users/4593c3fd1739432fa57f4a94c65cd299` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:01.826889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4593c3fd1739432fa57f4a94c65cd299)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:01.828783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:01.917528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:01.932893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80c0266ae9684c838b98a30035e256cd', 'request_id': 'req-f3264fcc-468c-4964-a5f0-d6421973626d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.991803 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80c0266ae9684c838b98a30035e256cd', 'request_id': 'req-f3264fcc-468c-4964-a5f0-d6421973626d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:01.999485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80c0266ae9684c838b98a30035e256cd', 'request_id': 'req-f3264fcc-468c-4964-a5f0-d6421973626d'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.006559 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4593c3fd1739432fa57f4a94c65cd299'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.007264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4593c3fd1739432fa57f4a94c65cd299'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.011097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] The token cache is being invalidate because project 80c0266ae9684c838b98a30035e256cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:02.011461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.012110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-f3264fcc-468c-4964-a5f0-d6421973626d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.013647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1123/2238] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:01 2022] DELETE /identity/v3/projects/80c0266ae9684c838b98a30035e256cd => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:02.018277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.018560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.018863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.022159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4593c3fd1739432fa57f4a94c65cd299'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.039100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:02.041031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.076675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4593c3fd1739432fa57f4a94c65cd299'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.077297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-38783bac-aada-4a52-b615-3abe994d722c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4593c3fd1739432fa57f4a94c65cd299'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.092176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1116/2239] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:01 2022] DELETE /identity/v3/users/4593c3fd1739432fa57f4a94c65cd299 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:02.107530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1124/2240] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:02.116638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.116963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.117246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.137998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:02.139248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.203345 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:02.213736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1117/2241] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:02.218213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.228802 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55872a20-976b-45f9-b071-f5f8a6d9493b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.234171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.234171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.234414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.234593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.235333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:02.235633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.237221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.250464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1125/2242] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:02.254195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.264241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef15a6-9362-43a5-b81d-52cf93845042 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.268918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.269512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.269786 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.270052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.270708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.272594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.353496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.355401 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.368707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8bc7ae-6729-4b46-81f6-30040cf65be1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.373443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.374078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.377070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.377070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] PATH_INFO: `/v3/projects/c83d10829a81437fb4d56fe488581212` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.377070 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c83d10829a81437fb4d56fe488581212)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.377863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.379543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1118/2243] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:02.384436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.451367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f62a9-f1ae-40c2-819c-801fdd74a772 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.456047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.456644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.456921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.457189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.457939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.459866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.500343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c83d10829a81437fb4d56fe488581212', 'request_id': 'req-176adf35-1649-45e8-942f-8f8dad6a7bd3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.625911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c83d10829a81437fb4d56fe488581212', 'request_id': 'req-176adf35-1649-45e8-942f-8f8dad6a7bd3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.633415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c83d10829a81437fb4d56fe488581212', 'request_id': 'req-176adf35-1649-45e8-942f-8f8dad6a7bd3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.645035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] The token cache is being invalidate because project c83d10829a81437fb4d56fe488581212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:02.645353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.646045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-176adf35-1649-45e8-942f-8f8dad6a7bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:02.647693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1126/2244] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:02 2022] DELETE /identity/v3/projects/c83d10829a81437fb4d56fe488581212 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:02.653050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125bf7-521d-4268-a3dc-e5418ae3b74f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.653400 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125bf7-521d-4268-a3dc-e5418ae3b74f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.653693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea125bf7-521d-4268-a3dc-e5418ae3b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.673137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ea125bf7-521d-4268-a3dc-e5418ae3b74f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:02.674322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ea125bf7-521d-4268-a3dc-e5418ae3b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.693639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1119/2245] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/users => generated 291 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:02.694914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.696595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.700386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:02.709521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefae0b2-ebae-44d3-bf9a-8f19761e780c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.735058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1127/2246] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:02.742605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.742989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.743273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.764385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:02.765497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:02.768924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.769539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.769821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.770093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.770765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.772361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.773407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1128/2247] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:02.778022 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.783039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1120/2248] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:02.787393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0ea29-9037-4bac-85e4-9648597ac17f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.792714 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.793316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.793593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.793691 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.793864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.794535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:02.794876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.796746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.804916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73de462-65c1-48d5-84b5-79b5eb085b48 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.810659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.811380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.811798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.812166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.812442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1129/2249] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:02.812992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.813494 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.814728 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.823538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cf32da-b847-4838-b298-459846d48c84 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.826673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1121/2250] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:02.830769 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.831590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.832189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.832470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.832741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.833366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.835295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.840949 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87526679-6a1e-42a1-a2d5-5ab6de71d625 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:02.845287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:02.845931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.846214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.846493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] PATH_INFO: `/v3/projects/0003d79ce5b0407a8baf4d5544fa7a90/users/a01f8f0919534a7b84dd8743c72c9824/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.861771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=0003d79ce5b0407a8baf4d5544fa7a90, user_id=a01f8f0919534a7b84dd8743c72c9824, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:02.864588 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f429d2e-8cf8-4390-8e3d-6d4e9ecae2ef admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:02.941798 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1130/2251] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:02.946068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:02.952702 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1122/2252] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:02 2022] PUT /identity/v3/projects/0003d79ce5b0407a8baf4d5544fa7a90/users/a01f8f0919534a7b84dd8743c72c9824/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:02.963351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:02.963351 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:02.963745 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:02.995580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] MFA Rules not processed for user `a01f8f0919534a7b84dd8743c72c9824`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:02.996806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:03.009160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:03.023970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db459b60-3efa-4522-a42c-307a2c2cf7ee admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.062047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1123/2253] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:03.066785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.077170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.077777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.078061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.078330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.078988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.081038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.129908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ec602-54cb-4c6b-8816-c35a00f8e831 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.134489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.135548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.135830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.136102 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.136737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.138652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.181488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1131/2254] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:02 2022] POST /identity/v3/users => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:03.185235 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.194569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c3eb4e-7569-4168-aa6d-cfb0ce239182 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.199367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.199937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.200215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.200483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.201120 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.202800 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.213047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1132/2255] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:03.217181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.226997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527686c-05e4-4664-afc0-36cae5c87c43 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.232513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.233073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.236181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.236181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.236181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.236181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.246651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1133/2256] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:03.250466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.255377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:03.261162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3621b6a7-c4fb-4010-a2ad-d7438e783ad2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.265699 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1124/2257] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/projects => generated 379 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:03.266546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.318982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.319561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.319838 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.320109 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] PATH_INFO: `/v3/projects/25ac1a73a8184cfeb7e3f6717760682d/users/af2a37335a7a4874b1e44f4b9714fee5/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.332136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef8a5d6-edd7-40a2-8940-0a559cb4e746 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.341525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25ac1a73a8184cfeb7e3f6717760682d, user_id=af2a37335a7a4874b1e44f4b9714fee5, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.341665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.341665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.341665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.341665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.341985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.342066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.342066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eef27a-68a9-4254-ba40-6d98dc35263e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.432477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1134/2258] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:03 2022] PUT /identity/v3/projects/25ac1a73a8184cfeb7e3f6717760682d/users/af2a37335a7a4874b1e44f4b9714fee5/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:03.436046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1125/2259] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/users => generated 296 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:03.436825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.445212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.445537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.445818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.481381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] MFA Rules not processed for user `af2a37335a7a4874b1e44f4b9714fee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:03.482667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:03.502990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:03.515613 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b6f4b-465e-46b1-9ce3-e56cc574006e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.520864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.521478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.521789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.522087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.522797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.524415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.538778 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1135/2260] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:03.539786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.550419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249d731-f852-4a50-b5a8-8770be8f1adc admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.555759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.556619 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.557268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.557268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] PATH_INFO: `/v3/projects/e7236f8dd325497086964cfe98927cf0/users/e95095a33ec94f808dc984874b7c0fec/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.560165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1126/2261] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:03.564472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.574541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7236f8dd325497086964cfe98927cf0, user_id=e95095a33ec94f808dc984874b7c0fec, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.578677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ece00-4f63-4acf-90ff-fe4e5b26baf9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.579115 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.582025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.582733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.583011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.583283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.583912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.585945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.672726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1136/2262] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:03 2022] PUT /identity/v3/projects/e7236f8dd325497086964cfe98927cf0/users/e95095a33ec94f808dc984874b7c0fec/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:03.676967 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.693686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1127/2263] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/projects => generated 403 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:03.698132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.753259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:03.758501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b7a7a9-1e03-484f-a37c-31c5076ca49c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.771160 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eff970-54dc-48fc-b1c7-436f5c503bab admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.776522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.777120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.777396 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.777667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.778324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.780314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.818676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.819298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.819581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.819855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.820504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.822596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.834396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1137/2264] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:03.836530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.847048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7304926e-10af-4955-b427-3cc0b8f95bf0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.851987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.852599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.852887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.858043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] PATH_INFO: `/v3/projects/e7236f8dd325497086964cfe98927cf0/users/e95095a33ec94f808dc984874b7c0fec/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.858043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7236f8dd325497086964cfe98927cf0, user_id=e95095a33ec94f808dc984874b7c0fec, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.858043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789049d-0c9d-40a4-bd86-3220214b0fe5 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.867937 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1138/2265] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:03 2022] PUT /identity/v3/projects/e7236f8dd325497086964cfe98927cf0/users/e95095a33ec94f808dc984874b7c0fec/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:03.878960 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.881112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.881112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.884237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1128/2266] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/users => generated 308 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:03.889675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.899155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d477-15ad-4b1f-a749-ca7dd85483d2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.908715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] MFA Rules not processed for user `e95095a33ec94f808dc984874b7c0fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:03.909940 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:03.957567 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.958180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.958463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:03.958767 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:03.959409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:03.961015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:03.972415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1129/2267] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:03.976825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:03.987493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af915ba3-ffe1-4a1c-a8d5-f32443c9b87b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:03.989363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1139/2268] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:03.993329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:03.994717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:03.995100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.004480 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] PATH_INFO: `/v3/projects/3061409f25204e139ffa93b06f2ea1de/users/6073861e7eb247fe9605fdcccfe39deb/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.006297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.007765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:04.021985 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3061409f25204e139ffa93b06f2ea1de, user_id=6073861e7eb247fe9605fdcccfe39deb, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.024759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.068237 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d1b6d-4359-4196-8ea1-822c6b061b2a None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.072592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.073148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.073424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.073695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.082692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.084201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.090405 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:04.094317 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1140/2269] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:04 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:04.095661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1130/2270] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:03 2022] PUT /identity/v3/projects/3061409f25204e139ffa93b06f2ea1de/users/6073861e7eb247fe9605fdcccfe39deb/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:04.100393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.170989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ada8a-5fcc-462a-be61-7587c0407b77 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.176707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.177339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.177693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.177977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.178625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.180298 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d49fc1b-d0fe-47e0-8359-96aac0d85a12 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.192106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1131/2271] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:04.195882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.209593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f975e-8426-4c83-b7f5-30434d2dae86 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.213941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.214598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.214983 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.215275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] PATH_INFO: `/v3/projects/3061409f25204e139ffa93b06f2ea1de/users/6073861e7eb247fe9605fdcccfe39deb/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.217279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3061409f25204e139ffa93b06f2ea1de, user_id=6073861e7eb247fe9605fdcccfe39deb, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.220020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.232922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1141/2272] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:04 2022] PUT /identity/v3/projects/3061409f25204e139ffa93b06f2ea1de/users/6073861e7eb247fe9605fdcccfe39deb/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:04.243187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.243475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.244315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.273144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] MFA Rules not processed for user `6073861e7eb247fe9605fdcccfe39deb`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:04.274459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:04.366165 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1132/2273] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:04 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:04.386497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.391471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:04.448119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5b4be4-d244-4997-ba64-9d52b498eab7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.457157 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.457821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.458094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.458358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.473643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.475172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.486030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1142/2274] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:04 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:04.632736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.694652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a56d31-7641-485d-b1f0-765a460cae08 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.699391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.705036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.705036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.705036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] PATH_INFO: `/v3/users/af2a37335a7a4874b1e44f4b9714fee5` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.705036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af2a37335a7a4874b1e44f4b9714fee5)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.705036 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.726477 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:04.745549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:04.756294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea21be-4f6c-4ae4-8c49-01d53721d35c service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:04.761012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:04.761622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:04.761897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:04.762164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] PATH_INFO: `/v3/users/a01f8f0919534a7b84dd8743c72c9824` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:04.763295 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a01f8f0919534a7b84dd8743c72c9824)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:04.765185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:04.914148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af2a37335a7a4874b1e44f4b9714fee5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:04.914751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af2a37335a7a4874b1e44f4b9714fee5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:04.927304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af2a37335a7a4874b1e44f4b9714fee5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:04.989642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a01f8f0919534a7b84dd8743c72c9824'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.002101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a01f8f0919534a7b84dd8743c72c9824'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.002808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a01f8f0919534a7b84dd8743c72c9824'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.010595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a01f8f0919534a7b84dd8743c72c9824'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.011273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-7d489d72-e50a-4c32-ad8c-185c467b4557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a01f8f0919534a7b84dd8743c72c9824'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.015031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af2a37335a7a4874b1e44f4b9714fee5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.015662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-736f39e8-a79c-41bb-afa2-1ee19d129d19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af2a37335a7a4874b1e44f4b9714fee5'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:05.029265 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1133/2275] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:04 2022] DELETE /identity/v3/users/af2a37335a7a4874b1e44f4b9714fee5 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:05.033620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.033906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.034185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.053191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:05.054441 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.078125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1143/2276] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:04 2022] DELETE /identity/v3/users/a01f8f0919534a7b84dd8743c72c9824 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:05.082348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.082638 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.082963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.102528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:05.104302 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.119233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1134/2277] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:05.128905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:05.130286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.166996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1144/2278] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:05.176817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:05.178277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.188531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8e033-22a4-45ce-a564-745ae552cea9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:05.198445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:05.236025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ebd5f6-e50a-4540-8c42-094ba86e1096 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:05.238517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:05.239222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.239503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.239711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:05.239830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.240284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.240561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.240833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.249364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:05.250673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:05.250801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:05.252138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:05.255816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:05.261536 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1135/2279] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:05.262438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1145/2280] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:05.845560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:05.847126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.849858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:05.851256 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:05.863055 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9dbc71-1230-4109-9dd1-53c8dc8d2ed7 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:05.865201 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fc6d47-34a4-442f-8f7e-b26bca924efe service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:05.868380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:05.869045 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.872814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.872814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] PATH_INFO: `/v3/projects/0003d79ce5b0407a8baf4d5544fa7a90` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.872814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0003d79ce5b0407a8baf4d5544fa7a90)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:05.873150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:05.873301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:05.873628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:05.873887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:05.874319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] PATH_INFO: `/v3/projects/25ac1a73a8184cfeb7e3f6717760682d` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:05.875643 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] RBAC: Authorizing `identity:delete_project(project_id=25ac1a73a8184cfeb7e3f6717760682d)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:05.877793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:05.999026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0003d79ce5b0407a8baf4d5544fa7a90', 'request_id': 'req-404fb42d-da71-4441-8b38-bd7097586dfb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.021832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25ac1a73a8184cfeb7e3f6717760682d', 'request_id': 'req-d97ba109-bec5-4570-9b96-7c989e07016e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.117207 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0003d79ce5b0407a8baf4d5544fa7a90', 'request_id': 'req-404fb42d-da71-4441-8b38-bd7097586dfb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.124483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0003d79ce5b0407a8baf4d5544fa7a90', 'request_id': 'req-404fb42d-da71-4441-8b38-bd7097586dfb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.136694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] The token cache is being invalidate because project 0003d79ce5b0407a8baf4d5544fa7a90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:06.136986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.137629 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.139242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1136/2281] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:05 2022] DELETE /identity/v3/projects/0003d79ce5b0407a8baf4d5544fa7a90 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:06.143072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:06.144410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:06.174689 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25ac1a73a8184cfeb7e3f6717760682d', 'request_id': 'req-d97ba109-bec5-4570-9b96-7c989e07016e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.184036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25ac1a73a8184cfeb7e3f6717760682d', 'request_id': 'req-d97ba109-bec5-4570-9b96-7c989e07016e'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.193049 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] The token cache is being invalidate because project 25ac1a73a8184cfeb7e3f6717760682d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:06.193332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.193987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.195735 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1146/2282] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:05 2022] DELETE /identity/v3/projects/25ac1a73a8184cfeb7e3f6717760682d => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:06.199563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:06.200921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:06.210427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fb42d-da71-4441-8b38-bd7097586dfb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:06.269306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ba109-bec5-4570-9b96-7c989e07016e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:06.269572 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:06.270105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:06.270367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:06.270655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] PATH_INFO: `/v3/users/e95095a33ec94f808dc984874b7c0fec` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:06.271774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95095a33ec94f808dc984874b7c0fec)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:06.273693 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:06.274186 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:06.274869 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:06.275163 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:06.276042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] PATH_INFO: `/v3/users/6073861e7eb247fe9605fdcccfe39deb` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:06.277215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6073861e7eb247fe9605fdcccfe39deb)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:06.279149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:06.496008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:06.506008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:06.507741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6073861e7eb247fe9605fdcccfe39deb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.507892 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95095a33ec94f808dc984874b7c0fec'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.508538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95095a33ec94f808dc984874b7c0fec'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.520589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6073861e7eb247fe9605fdcccfe39deb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.521253 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6073861e7eb247fe9605fdcccfe39deb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.521731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95095a33ec94f808dc984874b7c0fec'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.529043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6073861e7eb247fe9605fdcccfe39deb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.529703 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6073861e7eb247fe9605fdcccfe39deb'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.643791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95095a33ec94f808dc984874b7c0fec'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.644454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95095a33ec94f808dc984874b7c0fec'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:06.658460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1137/2283] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:06 2022] DELETE /identity/v3/users/e95095a33ec94f808dc984874b7c0fec => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:06.740971 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1147/2284] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:06 2022] DELETE /identity/v3/users/6073861e7eb247fe9605fdcccfe39deb => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:07.190877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.191972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.192873 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.193461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.282237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac418ab-b311-4815-bd68-583c7f4d51ee admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.287675 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.288354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.288663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.288948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] PATH_INFO: `/v3/projects/e7236f8dd325497086964cfe98927cf0` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.290131 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7236f8dd325497086964cfe98927cf0)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.292665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:07.295324 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33daf10a-f6ad-4de6-be37-8dc7927ad421 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.300561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.301210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.301523 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.323569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] PATH_INFO: `/v3/projects/3061409f25204e139ffa93b06f2ea1de` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.323569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3061409f25204e139ffa93b06f2ea1de)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.323569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:07.536327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7236f8dd325497086964cfe98927cf0', 'request_id': 'req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.536570 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3061409f25204e139ffa93b06f2ea1de', 'request_id': 'req-dfda5123-ca64-4857-81ca-47ff8f168053'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.643166 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3061409f25204e139ffa93b06f2ea1de', 'request_id': 'req-dfda5123-ca64-4857-81ca-47ff8f168053'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.649949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3061409f25204e139ffa93b06f2ea1de', 'request_id': 'req-dfda5123-ca64-4857-81ca-47ff8f168053'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.661863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] The token cache is being invalidate because project 3061409f25204e139ffa93b06f2ea1de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:07.662149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.662812 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.664271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1148/2285] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:07 2022] DELETE /identity/v3/projects/3061409f25204e139ffa93b06f2ea1de => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:07.665074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.666955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.718806 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:07.722999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7236f8dd325497086964cfe98927cf0', 'request_id': 'req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.729990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7236f8dd325497086964cfe98927cf0', 'request_id': 'req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.731045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfda5123-ca64-4857-81ca-47ff8f168053 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.736578 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.737205 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.737481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.737760 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] PATH_INFO: `/v3/users/6b03390675a04b03aeff99282bfd2e02` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.738907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b03390675a04b03aeff99282bfd2e02)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.740932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:07.741373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] The token cache is being invalidate because project e7236f8dd325497086964cfe98927cf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:07.741670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.742344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-85dd4e8c-1d33-4a73-a87f-edbc0e67ae25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.743854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1138/2286] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:07 2022] DELETE /identity/v3/projects/e7236f8dd325497086964cfe98927cf0 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:07.748661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bc79-67d9-4cbd-aff5-f3b373b90df9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.748952 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bc79-67d9-4cbd-aff5-f3b373b90df9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.749235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc5bc79-67d9-4cbd-aff5-f3b373b90df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.769861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-2fc5bc79-67d9-4cbd-aff5-f3b373b90df9 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:07.771113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2fc5bc79-67d9-4cbd-aff5-f3b373b90df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.779654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1139/2287] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:07.781077 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72550-11b8-444b-a786-052ced58526f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.781364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72550-11b8-444b-a786-052ced58526f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.781638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d72550-11b8-444b-a786-052ced58526f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.801889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-04d72550-11b8-444b-a786-052ced58526f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:07.803039 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-04d72550-11b8-444b-a786-052ced58526f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.810918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1140/2288] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:07.813409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3740f5-4d52-46c3-b9a5-306c6f0f6e39 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.813409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3740f5-4d52-46c3-b9a5-306c6f0f6e39 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.813409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3740f5-4d52-46c3-b9a5-306c6f0f6e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.831605 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ae3740f5-4d52-46c3-b9a5-306c6f0f6e39 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:07.833344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ae3740f5-4d52-46c3-b9a5-306c6f0f6e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.841215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1141/2289] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:07.842579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.842891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.843169 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.863714 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:07.864889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.873087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1142/2290] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:07.873987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.883159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf03604-4755-465a-8672-6a3bbbe0376b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.888080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.888639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.888918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.889189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.889809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:07.890109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.891706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:07.900964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:07.905510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1143/2291] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:07.905510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.909291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b03390675a04b03aeff99282bfd2e02'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.915046 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1446f9-61ed-4a0f-bcd3-2866bac45d9a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.920812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.925384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.925384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.925384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.925384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:07.925384 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.926059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b03390675a04b03aeff99282bfd2e02'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.926059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b03390675a04b03aeff99282bfd2e02'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.926322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:07.932993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b03390675a04b03aeff99282bfd2e02'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.933672 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b03390675a04b03aeff99282bfd2e02'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:07.936964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1144/2292] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:07.937879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:07.943476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:07.958209 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7acd83-97a1-43fd-abb9-da89e82fa87e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:07.964015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:07.964321 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:07.964602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:07.964908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:07.965575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:07.967548 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8138f-1f2f-4f49-a4b2-aae84638ab9d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.001975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1149/2293] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:07 2022] DELETE /identity/v3/users/6b03390675a04b03aeff99282bfd2e02 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:08.003415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.004512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.067412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1145/2294] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:07 2022] POST /identity/v3/projects => generated 405 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:08.068580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.068966 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.069142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a3cc4-3170-42fe-b775-5e5c3a3ea95c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.069416 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.091085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:08.092325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.127281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.128160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.128443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.128759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.129392 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.131355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.152007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1146/2295] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:08.152837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.162128 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135bd63-7cc4-4841-bede-877f9a52e3c1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.167186 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.167815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.168092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.168362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.168978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.170902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.198261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1150/2296] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:08.199264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.200811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.206815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:08.255786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d7e5d-0f47-45e4-9dcb-675796700f44 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.259429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.260732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.260867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.261091 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.305493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1147/2297] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/users => generated 303 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:08.306312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.316652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f8f6a9-e671-4441-86ea-e33e7460b922 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.323893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.323893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.324144 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.324250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.324250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.324412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.324741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.326256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.334247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1151/2298] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:08.340336 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.349922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934a358-9c6d-4f54-961c-f4b7fed0184f service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.355297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.358464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.358464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.358464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.358464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.358901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.370985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1152/2299] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:08.374084 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.383953 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5053-5ac9-4e34-9b6d-b52726d62428 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.389195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.389791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.390343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.390343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.391170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.392751 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.403419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1153/2300] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:08.408131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.416903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1148/2301] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:08.419066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c581d92-66fc-4377-9766-3490491102e9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.420304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.424605 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.425349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.425630 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.425907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] PATH_INFO: `/v3/projects/e6425e5b2baa42bcadfc208010752ef5/users/1de7bbcc0dce4a338dda7e9db7c6e5c1/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.433537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0941599f-b55f-4ef9-91a4-1c94b18fb441 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.438574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.439221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.439606 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.439771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.440417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.442006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.443310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6425e5b2baa42bcadfc208010752ef5, user_id=1de7bbcc0dce4a338dda7e9db7c6e5c1, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.445951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a65c-6047-4f3c-bcb3-7c6f609a02d4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.454061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1149/2302] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:08.459836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.469590 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeda7c3-c6e8-4a87-a1ac-2d2f0ad9758e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.474511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.475170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.475446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.475718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.476344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.477903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.488716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1150/2303] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:08.492432 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.503201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e136dd3-e33a-4afb-9580-01cd599509a2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.509330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.509330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.509330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.509615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] PATH_INFO: `/v3/projects/6c5b34f2026d469f85c774cf9012d0bf/users/c642f9d09ca94ec0b9d3110ace798456/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.525197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c5b34f2026d469f85c774cf9012d0bf, user_id=c642f9d09ca94ec0b9d3110ace798456, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.528147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d36a7f-ad37-4e12-bb2b-e8848bc299b4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.535543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1154/2304] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:08 2022] PUT /identity/v3/projects/e6425e5b2baa42bcadfc208010752ef5/users/1de7bbcc0dce4a338dda7e9db7c6e5c1/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:08.540359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.540644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.540917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.565197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:08.565197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.624159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1151/2305] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:08 2022] PUT /identity/v3/projects/6c5b34f2026d469f85c774cf9012d0bf/users/c642f9d09ca94ec0b9d3110ace798456/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:08.643594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1155/2306] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:08.645132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.645584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.646310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.646310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.658389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:08.677422 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] MFA Rules not processed for user `c642f9d09ca94ec0b9d3110ace798456`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:08.678670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.707188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079849ee-708e-4a07-9300-fc0e8e7bde7e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.711278 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:08.745666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1152/2307] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:08.756119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.756734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.757018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.757292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.758685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.761222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.820935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e400c978-38b1-4586-b4b2-2fbcbb5284bb None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.822050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.824077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.825614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fccd7b631a914ab684da6d642501cb64', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.826225 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.826502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.826840 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.837491 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.838808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1156/2308] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:08.839745 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:08.839964 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.841380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:08.850563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1153/2309] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:08.860333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558582e5-283d-4e02-b6d6-34f3daf2594f service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:08.863336 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:08.864065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:08.864464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:08.864899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] PATH_INFO: `/v3/projects/73f58dbdf1c745ea81b943c73a400955` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:08.866135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f58dbdf1c745ea81b943c73a400955)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:08.868143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:08.997259 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f58dbdf1c745ea81b943c73a400955', 'request_id': 'req-c574633a-4ef4-4337-ad2e-16b75d531530'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.083330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f58dbdf1c745ea81b943c73a400955', 'request_id': 'req-c574633a-4ef4-4337-ad2e-16b75d531530'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.091134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f58dbdf1c745ea81b943c73a400955', 'request_id': 'req-c574633a-4ef4-4337-ad2e-16b75d531530'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.097793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:09.102152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] The token cache is being invalidate because project 73f58dbdf1c745ea81b943c73a400955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:09.102513 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.103266 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.104848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1157/2310] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:08 2022] DELETE /identity/v3/projects/73f58dbdf1c745ea81b943c73a400955 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:09.108911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:09.121167 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc531b8-2c22-420f-af89-9cdf92323915 service glance] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:09.184097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:09.184843 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:09.185195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:09.185539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] PATH_INFO: `/v3/users/98275bd2a53048628bf1ab808c7fc3f6` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:09.186795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98275bd2a53048628bf1ab808c7fc3f6)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:09.188752 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:09.418563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98275bd2a53048628bf1ab808c7fc3f6'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.419504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98275bd2a53048628bf1ab808c7fc3f6'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.441821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98275bd2a53048628bf1ab808c7fc3f6'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.540827 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98275bd2a53048628bf1ab808c7fc3f6'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.541963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a8a6152a-20c3-4ef8-9782-91ccf7736be2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98275bd2a53048628bf1ab808c7fc3f6'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:09.549410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:09.558974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1154/2311] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:09 2022] DELETE /identity/v3/users/98275bd2a53048628bf1ab808c7fc3f6 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:10.062610 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.064197 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.077657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c574633a-4ef4-4337-ad2e-16b75d531530 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.164451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.165347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.165627 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.165894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] PATH_INFO: `/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.167660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0ddf09a05e4c96b784ad7215fa1d08)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.170358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.246805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.247377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.253613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.288562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] MFA Rules not processed for user `1de7bbcc0dce4a338dda7e9db7c6e5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:10.288908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0ddf09a05e4c96b784ad7215fa1d08', 'request_id': 'req-351681c6-d32c-4ecf-846a-64e1af01f149'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:10.289804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.350868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0ddf09a05e4c96b784ad7215fa1d08', 'request_id': 'req-351681c6-d32c-4ecf-846a-64e1af01f149'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:10.358791 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0ddf09a05e4c96b784ad7215fa1d08', 'request_id': 'req-351681c6-d32c-4ecf-846a-64e1af01f149'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:10.370825 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1155/2312] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:10.372902 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.376829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.381684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] The token cache is being invalidate because project ef0ddf09a05e4c96b784ad7215fa1d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:10.381982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:10.382659 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-351681c6-d32c-4ecf-846a-64e1af01f149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:10.384161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1158/2313] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:10 2022] DELETE /identity/v3/projects/ef0ddf09a05e4c96b784ad7215fa1d08 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:10.389328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7613d8-ca40-4449-922f-f0ded2b0a56d None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.389882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7613d8-ca40-4449-922f-f0ded2b0a56d None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.390165 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7613d8-ca40-4449-922f-f0ded2b0a56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.410062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-af7613d8-ca40-4449-922f-f0ded2b0a56d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:10.411306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-af7613d8-ca40-4449-922f-f0ded2b0a56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.448408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3754b3c-7f12-4d5e-88b7-c8c0b7c99699 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.453555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.456035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.456035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.456035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.456035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.457389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.475723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1159/2314] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:10.483919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.484200 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.484472 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.505255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:10.506498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.514828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1160/2315] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:10.519008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.528577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8368-cb4e-471c-8618-d2374d54b8a8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.533229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.533810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.534092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.534364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.535035 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:10.535337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.536911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.549333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1161/2316] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:10.553415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.563506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf307-ecb0-46ce-88e8-816f6ef788fb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.573058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.576180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.580359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.580725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1156/2317] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/projects => generated 405 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:10.580866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.580866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.580866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.583141 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.584935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:10.586169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:10.650337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b34db0-4dc8-46bb-86c9-9b684620a20f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.656144 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.656859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.657139 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.657409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.658079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.662032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.696833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1162/2318] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/projects => generated 371 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:10.701072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.769931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba8f93-a5e1-48de-9ae0-2fa019999945 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.770223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:10.775670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1157/2319] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/users => generated 309 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:10.775793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.776435 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.776717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.776990 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.778485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.780196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.780637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.790537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758d620f-2824-4c95-9a25-a86618020d73 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.796602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.797227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.797508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.797781 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.798437 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.800085 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.810851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1158/2320] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:10.814974 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.824545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae61994-79f2-40dd-b244-9207595be130 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.829223 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.829839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.830166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.830443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] PATH_INFO: `/v3/projects/49cea33e695643e2b9d261add6a8a552/users/fd7e2ce29dce46ed840c2f0c10b25def/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.845289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cea33e695643e2b9d261add6a8a552, user_id=fd7e2ce29dce46ed840c2f0c10b25def, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.848089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.929471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1163/2321] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:10 2022] POST /identity/v3/users => generated 286 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:10.933683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.945840 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e72c4bf-caa9-422e-8a58-e798071cf7ca admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:10.955593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:10.955593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:10.955593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:10.955593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:10.956113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:10.956113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:10.959196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1159/2322] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:10 2022] PUT /identity/v3/projects/49cea33e695643e2b9d261add6a8a552/users/fd7e2ce29dce46ed840c2f0c10b25def/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:10.963034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.968274 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1164/2323] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:10.971865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:10.985064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8efbc1-f864-41c5-b10a-4e48a337c470 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.041023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc129919-a045-4235-b1b1-65c70437905f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.046780 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.047527 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.047657 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.047809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.048079 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.048269 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.048581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.048751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.048903 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.049482 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.050338 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.051092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.060861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1160/2324] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:11.062673 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1165/2325] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:11.066355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.066505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.076561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bef05-ef00-45b5-978e-51b624172817 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.076738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322982-f910-4787-aea7-1befc5568a43 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.081195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.081412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.081832 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.082054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.082162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.082336 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.082450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] PATH_INFO: `/v3/projects/0b08cfba031a4f64a1e67a5ff7f4373f/users/ffdf339bc89143a0afa0fcabc04f300a/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.082614 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] PATH_INFO: `/v3/projects/49cea33e695643e2b9d261add6a8a552/users/fd7e2ce29dce46ed840c2f0c10b25def/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.084571 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49cea33e695643e2b9d261add6a8a552, user_id=fd7e2ce29dce46ed840c2f0c10b25def, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.087269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338d0273-a03e-4d96-93b4-3f581aaa8341 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.098602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1161/2326] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:11 2022] PUT /identity/v3/projects/49cea33e695643e2b9d261add6a8a552/users/fd7e2ce29dce46ed840c2f0c10b25def/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:11.098820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b08cfba031a4f64a1e67a5ff7f4373f, user_id=ffdf339bc89143a0afa0fcabc04f300a, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.101556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.175532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1166/2327] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:11 2022] PUT /identity/v3/projects/0b08cfba031a4f64a1e67a5ff7f4373f/users/ffdf339bc89143a0afa0fcabc04f300a/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:11.186072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.191123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.191123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.218977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] MFA Rules not processed for user `ffdf339bc89143a0afa0fcabc04f300a`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:11.221047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:11.291451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1162/2328] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:11.296934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.340949 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:11.361181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a1e7e-d6f4-4918-b2ae-61bacb224702 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.365939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.366543 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.366850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.367124 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.367764 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.369681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.486018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1167/2329] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:11 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:11.490037 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.501708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe603fce-6d6d-4cbd-ad7f-259b080ac58f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.539931 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:11.563498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.564259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.564541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.564859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.565514 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.567510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.673122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1163/2330] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:11 2022] POST /identity/v3/users => generated 291 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:11.677561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.690011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e553b1-55be-4de1-ab9f-1f194ca35f64 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.694320 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.694968 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.695243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.695576 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.696202 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.697794 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.713010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1168/2331] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:11.719910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.734841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdabc7-5829-441c-acc7-3af5516d091b admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.739754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.740569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.740852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.741162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] PATH_INFO: `/v3/projects/b00c6e9860e5406c90895022d79aedb8/users/e0fba654ce3f4332b71abe0db36a5405/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.757524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00c6e9860e5406c90895022d79aedb8, user_id=e0fba654ce3f4332b71abe0db36a5405, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.760196 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.856415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1164/2332] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:11 2022] PUT /identity/v3/projects/b00c6e9860e5406c90895022d79aedb8/users/e0fba654ce3f4332b71abe0db36a5405/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:11.860671 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.875542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc5765-c5cd-4535-b8e2-736415a3d2b3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:11.968138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:11.968753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:11.969033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:11.969307 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:11.970004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:11.971574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a63648-d147-4c54-9278-826f3c09eac7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:11.981965 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1169/2333] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:11.986224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:11.997092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6a193-a57e-4635-9c79-152c55168be8 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:12.001631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:12.002268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:12.002545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:12.002851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] PATH_INFO: `/v3/projects/b00c6e9860e5406c90895022d79aedb8/users/e0fba654ce3f4332b71abe0db36a5405/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:12.012498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b00c6e9860e5406c90895022d79aedb8, user_id=e0fba654ce3f4332b71abe0db36a5405, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:12.015408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:12.038861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1165/2334] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:11 2022] PUT /identity/v3/projects/b00c6e9860e5406c90895022d79aedb8/users/e0fba654ce3f4332b71abe0db36a5405/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:12.050646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e56ca-ef51-4758-a5cf-9b747192df8f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:12.050944 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e56ca-ef51-4758-a5cf-9b747192df8f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:12.051306 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e56ca-ef51-4758-a5cf-9b747192df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:12.091413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-440e56ca-ef51-4758-a5cf-9b747192df8f None None] MFA Rules not processed for user `e0fba654ce3f4332b71abe0db36a5405`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:12.092651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-440e56ca-ef51-4758-a5cf-9b747192df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:12.188973 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1170/2335] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:12.200899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:12.202673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:12.272355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffbc53-8034-4c1d-b769-b9d4f6ba4ba6 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:12.276100 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:12.276694 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:12.276982 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:12.277318 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:12.286391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:12.288058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:12.298150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1166/2336] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:12.423014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb4f7-4918-48a3-95fe-34e118b4f08b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:12.423699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb4f7-4918-48a3-95fe-34e118b4f08b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:12.424177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6eb4f7-4918-48a3-95fe-34e118b4f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:12.437651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:12.459162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-ca6eb4f7-4918-48a3-95fe-34e118b4f08b None None] MFA Rules not processed for user `fd7e2ce29dce46ed840c2f0c10b25def`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:12.467092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-ca6eb4f7-4918-48a3-95fe-34e118b4f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:12.553211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1171/2337] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:13.239180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:13.323939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:13.337575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d8a24-c4a8-4bf3-8a8e-b7c80cbae1d1 service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:13.342412 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:13.343105 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:13.343383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:13.343684 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] PATH_INFO: `/v3/users/ffdf339bc89143a0afa0fcabc04f300a` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:13.344898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffdf339bc89143a0afa0fcabc04f300a)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:13.346861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:13.516887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffdf339bc89143a0afa0fcabc04f300a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:13.517635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffdf339bc89143a0afa0fcabc04f300a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:13.532173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffdf339bc89143a0afa0fcabc04f300a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:13.637634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffdf339bc89143a0afa0fcabc04f300a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:13.638327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffdf339bc89143a0afa0fcabc04f300a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:13.651129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1167/2338] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:13 2022] DELETE /identity/v3/users/ffdf339bc89143a0afa0fcabc04f300a => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:13.656352 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:13.656836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:13.656951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:13.685787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:13.687036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:13.758742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1172/2339] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:13.768586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:13.770158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:13.838211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c68ada6-6f5b-476d-9465-1e4c0f93fc29 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:13.843663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:13.844537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:13.844887 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:13.845283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:13.854847 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:13.856915 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:13.866299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1168/2340] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:13.970531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:13.972099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:13.987413 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd662ccd-eab6-4f0e-8d1e-e5494d1f3db1 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:13.992285 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:13.997019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:13.997019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:13.997019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] PATH_INFO: `/v3/users/1de7bbcc0dce4a338dda7e9db7c6e5c1` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:13.997019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1de7bbcc0dce4a338dda7e9db7c6e5c1)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:13.997440 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:14.200749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1de7bbcc0dce4a338dda7e9db7c6e5c1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.212995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1de7bbcc0dce4a338dda7e9db7c6e5c1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.213700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1de7bbcc0dce4a338dda7e9db7c6e5c1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.221608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1de7bbcc0dce4a338dda7e9db7c6e5c1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.222255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1de7bbcc0dce4a338dda7e9db7c6e5c1'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.279385 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:14.284607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1173/2341] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:13 2022] DELETE /identity/v3/users/1de7bbcc0dce4a338dda7e9db7c6e5c1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:14.376581 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:14.378098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:14.397483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df31241-675d-4039-bf4f-c8cc35341aad service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:14.472075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:14.472713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:14.472989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:14.473257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] PATH_INFO: `/v3/projects/0b08cfba031a4f64a1e67a5ff7f4373f` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:14.474380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b08cfba031a4f64a1e67a5ff7f4373f)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:14.476319 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:14.478620 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:14.515617 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:14.517128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:14.530889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33dd999-19c6-415a-8190-0c75c40910eb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:14.535872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:14.536515 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:14.536796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:14.537171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] PATH_INFO: `/v3/projects/e6425e5b2baa42bcadfc208010752ef5` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:14.538376 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6425e5b2baa42bcadfc208010752ef5)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:14.540367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:14.561772 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b08cfba031a4f64a1e67a5ff7f4373f', 'request_id': 'req-eb3529d5-83ee-418c-8591-67e2d46cf0e7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.631733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b08cfba031a4f64a1e67a5ff7f4373f', 'request_id': 'req-eb3529d5-83ee-418c-8591-67e2d46cf0e7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.640895 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b08cfba031a4f64a1e67a5ff7f4373f', 'request_id': 'req-eb3529d5-83ee-418c-8591-67e2d46cf0e7'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.650041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] The token cache is being invalidate because project 0b08cfba031a4f64a1e67a5ff7f4373f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:14.650331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.650993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.652508 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1169/2342] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:14 2022] DELETE /identity/v3/projects/0b08cfba031a4f64a1e67a5ff7f4373f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:14.656457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:14.657725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:14.708322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6425e5b2baa42bcadfc208010752ef5', 'request_id': 'req-c400163c-d6dd-41fb-bdad-7d534973491f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.724537 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3529d5-83ee-418c-8591-67e2d46cf0e7 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:14.780539 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:14.781166 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:14.781452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:14.781719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] PATH_INFO: `/v3/users/e0fba654ce3f4332b71abe0db36a5405` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:14.782886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0fba654ce3f4332b71abe0db36a5405)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:14.784978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:14.792060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6425e5b2baa42bcadfc208010752ef5', 'request_id': 'req-c400163c-d6dd-41fb-bdad-7d534973491f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.800630 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6425e5b2baa42bcadfc208010752ef5', 'request_id': 'req-c400163c-d6dd-41fb-bdad-7d534973491f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.815348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] The token cache is being invalidate because project e6425e5b2baa42bcadfc208010752ef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:14.815646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.816303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:14.817801 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1174/2343] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:14 2022] DELETE /identity/v3/projects/e6425e5b2baa42bcadfc208010752ef5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:14.827003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:14.828418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:14.841796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400163c-d6dd-41fb-bdad-7d534973491f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:14.846389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:14.847044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:14.847349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:14.847589 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] PATH_INFO: `/v3/users/fd7e2ce29dce46ed840c2f0c10b25def` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:14.848700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7e2ce29dce46ed840c2f0c10b25def)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:14.850598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:15.019324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:15.030393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0fba654ce3f4332b71abe0db36a5405'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.031097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0fba654ce3f4332b71abe0db36a5405'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.044163 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0fba654ce3f4332b71abe0db36a5405'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.074339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7e2ce29dce46ed840c2f0c10b25def'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.086618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7e2ce29dce46ed840c2f0c10b25def'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.087316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7e2ce29dce46ed840c2f0c10b25def'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.094945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7e2ce29dce46ed840c2f0c10b25def'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.095579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7e2ce29dce46ed840c2f0c10b25def'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.117312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0fba654ce3f4332b71abe0db36a5405'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.117916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0fba654ce3f4332b71abe0db36a5405'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.131123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1170/2344] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:14 2022] DELETE /identity/v3/users/e0fba654ce3f4332b71abe0db36a5405 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:15.163374 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1175/2345] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:14 2022] DELETE /identity/v3/users/fd7e2ce29dce46ed840c2f0c10b25def => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:15.444041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:15.445751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:15.536543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa30dd0-e9ad-4388-afab-1efedef83b6e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:15.592117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:15.592625 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:15.592901 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:15.593173 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] PATH_INFO: `/v3/projects/49cea33e695643e2b9d261add6a8a552` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:15.594280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=49cea33e695643e2b9d261add6a8a552)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:15.596280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:15.599538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:15.603908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:15.620531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d205dc2-6db6-4a2f-8b30-eff9b2984027 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:15.625957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:15.627548 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:15.627676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:15.627916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] PATH_INFO: `/v3/projects/b00c6e9860e5406c90895022d79aedb8` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:15.629100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=b00c6e9860e5406c90895022d79aedb8)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:15.631101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:15.745313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49cea33e695643e2b9d261add6a8a552', 'request_id': 'req-900da88e-c85a-4854-a628-d6843c6c9ecd'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.748018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=77,keystone:DELETE=21,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:15.750127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b00c6e9860e5406c90895022d79aedb8', 'request_id': 'req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.819410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49cea33e695643e2b9d261add6a8a552', 'request_id': 'req-900da88e-c85a-4854-a628-d6843c6c9ecd'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.825377 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49cea33e695643e2b9d261add6a8a552', 'request_id': 'req-900da88e-c85a-4854-a628-d6843c6c9ecd'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.831203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:15.837007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] The token cache is being invalidate because project 49cea33e695643e2b9d261add6a8a552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:15.837296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.837963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-900da88e-c85a-4854-a628-d6843c6c9ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.839494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1171/2346] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:15 2022] DELETE /identity/v3/projects/49cea33e695643e2b9d261add6a8a552 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:15.844711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5b2a-3846-45e2-bfd6-836762c0c7d7 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:15.845031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5b2a-3846-45e2-bfd6-836762c0c7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:15.845264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f5b2a-3846-45e2-bfd6-836762c0c7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:15.876333 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b00c6e9860e5406c90895022d79aedb8', 'request_id': 'req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.883178 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b00c6e9860e5406c90895022d79aedb8', 'request_id': 'req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.893562 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] The token cache is being invalidate because project b00c6e9860e5406c90895022d79aedb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:15.893848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.894471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-3847353f-3a68-4dcf-ae4a-da1cf4f5c2fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:15.895970 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1176/2347] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:15 2022] DELETE /identity/v3/projects/b00c6e9860e5406c90895022d79aedb8 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:15.899556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-381f5b2a-3846-45e2-bfd6-836762c0c7d7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:15.900598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463220e9-05d1-4349-9eec-8a9775d9f6e6 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:15.900849 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-381f5b2a-3846-45e2-bfd6-836762c0c7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:15.900985 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463220e9-05d1-4349-9eec-8a9775d9f6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:15.901138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463220e9-05d1-4349-9eec-8a9775d9f6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:15.923916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-463220e9-05d1-4349-9eec-8a9775d9f6e6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:15.925148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-463220e9-05d1-4349-9eec-8a9775d9f6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:15.979464 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1172/2348] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:15.988087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:15.988545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:15.988783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.008850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1177/2349] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:16.010890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:16.012198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:16.015611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.015931 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.016349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.024687 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1173/2350] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:16.026864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.040323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec66ba-afd3-4216-8557-409c4809cbab None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.042793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:16.044608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:16.052870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1178/2351] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:16.053401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.054072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.054417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.054789 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.055511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:16.056148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.056929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.057785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.066550 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5089ff-f5e8-4d3e-9a0a-863d63cc5b94 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.071679 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1174/2352] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:16.071836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.071955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.077736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.077736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.077736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:16.077736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.077736 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.080771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.087726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1179/2353] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:16.091447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c936f10-3097-481d-8f35-3366f6901940 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.092571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.096467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.097067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.097343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.097612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.098277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.100301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.106411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43166f4a-915e-4970-823b-a3bf75d1abd6 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.108169 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.108757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.109036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.109478 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.110112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.112388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.213343 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1175/2354] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:16.214360 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.216312 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1180/2355] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/projects => generated 375 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:16.220085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.279990 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d225ddb1-f351-4042-a544-9165313c6a9d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.284971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.285142 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404e7ce8-f23b-4ca1-b2ae-01be615bbdff admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.285705 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.285987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.286257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.286918 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.288971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.289710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.290292 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.290569 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.290863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.291542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.294982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.402324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1176/2356] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/users => generated 285 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:16.406653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.422603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdc69dc-df17-4e65-9956-526bca155e70 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.428097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1181/2357] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/users => generated 288 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:16.432131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.437080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.437723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.438010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.438287 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.439262 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.441381 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.444951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc6d3e9-47a8-4ad7-8272-a03a416f4fb2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.450582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.454921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.454921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.454921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.454921 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.455292 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1177/2358] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:16.455787 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.457654 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.466249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1182/2359] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:16.467989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129dd76-ec27-4213-af53-2b76e13140ca admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.470276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.473108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.473819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.474089 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.474357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.475313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.478093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.478543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:16.480134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267917ca-6bb4-42e2-960a-b729ad28ab0d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.485530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.486220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.486498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.486813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.487452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.489023 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.493069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1178/2360] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:16.494372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.503031 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1183/2361] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:16.505267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c783-b532-4938-8651-982b10d99bdd admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.506143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.509721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.510373 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.510673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.510957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] PATH_INFO: `/v3/projects/ef5ba7e430554c58bd169a775fa06ef2/users/c317406671f2493ea000d7e31241bee3/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.511390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:16.517162 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ba33f-cabc-41d8-b2e0-4376b5c0b1c2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.521819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.522467 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.522770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.523040 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] PATH_INFO: `/v3/projects/80378bd364cf43018a211197fdf30ee4/users/a9045eaeb805432da2112031ad92c74f/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.525948 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef5ba7e430554c58bd169a775fa06ef2, user_id=c317406671f2493ea000d7e31241bee3, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.528924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9c49-e82e-4839-bc60-b71fdb8da593 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.538244 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80378bd364cf43018a211197fdf30ee4, user_id=a9045eaeb805432da2112031ad92c74f, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.541047 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173802-f034-4ba1-84e3-32a7f9ed0e78 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.626034 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1179/2362] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:16 2022] PUT /identity/v3/projects/ef5ba7e430554c58bd169a775fa06ef2/users/c317406671f2493ea000d7e31241bee3/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:16.637771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.638057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.638339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.661977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1184/2363] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:16 2022] PUT /identity/v3/projects/80378bd364cf43018a211197fdf30ee4/users/a9045eaeb805432da2112031ad92c74f/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:16.672242 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] MFA Rules not processed for user `c317406671f2493ea000d7e31241bee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:16.673836 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:16.674212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.674583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.674890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.702190 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] MFA Rules not processed for user `a9045eaeb805432da2112031ad92c74f`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:16.703421 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:16.739790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1180/2364] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:16.744568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.772064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1185/2365] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:16.776708 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.811145 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f69fbf-e949-4289-a64a-1c89f4181ee9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.815663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.816579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.816579 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.816798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.817402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.819327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.840784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf1225-a0b1-4930-b4ed-170c8671c226 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:16.845441 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:16.847556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:16.847556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:16.847556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:16.847556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:16.849281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:16.930822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1181/2366] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:16.935410 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.955062 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1186/2367] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:16.955986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:16.988363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:17.003339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1816d343-f899-4fc5-a27d-9058799d4034 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.019827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:17.032247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189222a6-c4ba-4acb-94a1-2b97c8c8ed28 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.037098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.037700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.037977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.038260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.038933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.040961 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.070855 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.071495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.071778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.072048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.072677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.075463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.123615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1187/2368] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/users => generated 295 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:17.127329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.138314 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cfde80-4b6a-4a9b-9aaa-0a806d732da3 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.142763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1182/2369] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:16 2022] POST /identity/v3/users => generated 291 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:17.143557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.146902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.146902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.146902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.146902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.147280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.147828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.162075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1188/2370] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:17.162280 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02fee8-d6d1-43fd-b7b3-74b3ac15d042 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.163273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.170584 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.171230 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.171768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.172042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.172688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.173701 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6d587-7cac-4101-81c4-5c0d4feca59d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.174291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.178453 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.179221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.179496 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.179780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] PATH_INFO: `/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594/users/abb44e97e37d406297f13b9e34734dfd/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.185109 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1183/2371] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:17.189325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.198338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a3b4c768ec42db83d678d9fa3c1594, user_id=abb44e97e37d406297f13b9e34734dfd, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.202717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.203865 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd30dc6-7755-48a8-a819-0bf66fce3744 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.209589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.210179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.210483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.210798 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] PATH_INFO: `/v3/projects/6bad303d7d484d9abbee20a19c3cb020/users/62fa72a7ff794339863b22473f595e23/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.228612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bad303d7d484d9abbee20a19c3cb020, user_id=62fa72a7ff794339863b22473f595e23, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.231465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.289388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1189/2372] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:17 2022] PUT /identity/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594/users/abb44e97e37d406297f13b9e34734dfd/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:17.293590 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.322814 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1184/2373] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:17 2022] PUT /identity/v3/projects/6bad303d7d484d9abbee20a19c3cb020/users/62fa72a7ff794339863b22473f595e23/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:17.326228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.358920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57ee758-4140-49cb-9c51-c7b57a89dee9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.401361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bd0e2-daa7-4008-a34e-4ca6be572e49 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.412409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.413266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.413660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.414030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.415227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.418035 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.429445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.430380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.430846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.431277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.432294 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.435544 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.436531 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1185/2374] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:17.440526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.448849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1190/2375] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:17.451436 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ba207f-c586-4e5c-8209-50461e275110 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.454282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.457710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.458370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.458662 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.458935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] PATH_INFO: `/v3/projects/6bad303d7d484d9abbee20a19c3cb020/users/62fa72a7ff794339863b22473f595e23/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.460867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bad303d7d484d9abbee20a19c3cb020, user_id=62fa72a7ff794339863b22473f595e23, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.464010 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aa9a07-4e83-4aca-a95d-dede4f9b952a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.464335 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bcd373-8293-4aa1-ae62-2e3979fc3c98 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.469684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.470141 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.470419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.470728 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] PATH_INFO: `/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594/users/abb44e97e37d406297f13b9e34734dfd/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.472664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6a3b4c768ec42db83d678d9fa3c1594, user_id=abb44e97e37d406297f13b9e34734dfd, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.475318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbca1eb8-994b-464e-a328-8315c3128734 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.476114 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1186/2376] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:17 2022] PUT /identity/v3/projects/6bad303d7d484d9abbee20a19c3cb020/users/62fa72a7ff794339863b22473f595e23/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:17.495380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.495380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.495380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.495629 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1191/2377] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:17 2022] PUT /identity/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594/users/abb44e97e37d406297f13b9e34734dfd/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:17.504493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.504784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.505067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.518346 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] MFA Rules not processed for user `62fa72a7ff794339863b22473f595e23`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:17.519607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:17.534351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] MFA Rules not processed for user `abb44e97e37d406297f13b9e34734dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:17.535572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:17.539286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:17.564549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:17.599957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1187/2378] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:17.609063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.610459 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:17.619867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1192/2379] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:17.634652 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.636113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:17.671391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564b262-da60-4905-926b-db57c1e4aa36 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.675476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.676093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.676374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.676647 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.686168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.687635 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.695451 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f818dc8-97a0-435d-8bb4-e58f6069fe59 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.697203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1188/2380] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:17.699639 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.700261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.700537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.700810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.758131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.759603 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.769177 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1193/2381] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:17.821178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:17.830754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e045b6-5852-4160-8015-c875ff82081c service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:17.834740 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:17.835545 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:17.835821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:17.836092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:17.900914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:17.900914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e1e45-e222-4c2b-907f-cbec9c921be3 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:17.907790 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1189/2382] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:22.616422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:22.677682 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1cd45-1c81-4b70-a898-0c3c5130b91f service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:22.682587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:22.683849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:22.684161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:22.684432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] PATH_INFO: `/v3/users/c317406671f2493ea000d7e31241bee3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:22.685681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c317406671f2493ea000d7e31241bee3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:22.687788 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:22.709092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:22.861713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c317406671f2493ea000d7e31241bee3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:22.874048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c317406671f2493ea000d7e31241bee3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:22.874828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c317406671f2493ea000d7e31241bee3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:22.881757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c317406671f2493ea000d7e31241bee3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:22.882473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c317406671f2493ea000d7e31241bee3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:22.969586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1194/2383] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:22 2022] DELETE /identity/v3/users/c317406671f2493ea000d7e31241bee3 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:22.974016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0876-3006-463a-b941-557d78900f19 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:22.974509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0876-3006-463a-b941-557d78900f19 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:22.974608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a0876-3006-463a-b941-557d78900f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:22.995561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-097a0876-3006-463a-b941-557d78900f19 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:22.997430 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-097a0876-3006-463a-b941-557d78900f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.045986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:23.055300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1190/2384] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:23.065665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:23.067082 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.125886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ab37c-f872-4797-89ae-23f5b271c96d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:23.129635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:23.130239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:23.130514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:23.132604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:23.141159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:23.142583 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:23.154606 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1195/2385] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:23.161330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a0876-3006-463a-b941-557d78900f19 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:23.161330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-097a0876-3006-463a-b941-557d78900f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.252389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a0876-3006-463a-b941-557d78900f19 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:23.257729 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:23.258362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:23.258665 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:23.258943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:23.260673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.273290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:23.274939 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:23.286013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1191/2386] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:23.616146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:23.617564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.635695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8402cc-bf2f-48f6-b491-46d079e009d0 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:23.642138 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:23.642899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:23.643088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:23.643341 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] PATH_INFO: `/v3/projects/ef5ba7e430554c58bd169a775fa06ef2` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:23.644575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef5ba7e430554c58bd169a775fa06ef2)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:23.646476 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:23.734651 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef5ba7e430554c58bd169a775fa06ef2', 'request_id': 'req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:23.813499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef5ba7e430554c58bd169a775fa06ef2', 'request_id': 'req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:23.820749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef5ba7e430554c58bd169a775fa06ef2', 'request_id': 'req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:23.832017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] The token cache is being invalidate because project ef5ba7e430554c58bd169a775fa06ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:23.832328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:23.832974 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:23.834539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1196/2387] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:23 2022] DELETE /identity/v3/projects/ef5ba7e430554c58bd169a775fa06ef2 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:23.838519 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:23.849257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625972c8-965e-4d21-9650-d7443871e86f service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:23.906291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:23.906854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:23.907121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:23.907388 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] PATH_INFO: `/v3/users/62fa72a7ff794339863b22473f595e23` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:23.908832 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fa72a7ff794339863b22473f595e23)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:23.910758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:23.967382 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:23.968769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:23.998667 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:24.034337 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5918b086-2ee2-4b43-a2d6-c7a88bbda4c0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:24.038019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:24.038622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:24.038932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:24.039199 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:24.040895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:24.103872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:24.106855 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:24.117568 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1197/2388] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:24.126998 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fa72a7ff794339863b22473f595e23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.127955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fa72a7ff794339863b22473f595e23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.143692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fa72a7ff794339863b22473f595e23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.148157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:24.214602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fa72a7ff794339863b22473f595e23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.215271 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-13417ba3-a619-4668-87a8-e0692098342b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fa72a7ff794339863b22473f595e23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.227875 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1192/2389] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:23 2022] DELETE /identity/v3/users/62fa72a7ff794339863b22473f595e23 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:24.599076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:24.600528 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:24.616121 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42458f6c-b7f3-48b5-9140-65edd863bcb7 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:24.690874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:24.690874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:24.690874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:24.690874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] PATH_INFO: `/v3/projects/6bad303d7d484d9abbee20a19c3cb020` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:24.691733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bad303d7d484d9abbee20a19c3cb020)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:24.691733 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:24.798445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bad303d7d484d9abbee20a19c3cb020', 'request_id': 'req-84897577-2f47-4ae3-9ca4-39b84ae06a98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.865328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bad303d7d484d9abbee20a19c3cb020', 'request_id': 'req-84897577-2f47-4ae3-9ca4-39b84ae06a98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.872954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bad303d7d484d9abbee20a19c3cb020', 'request_id': 'req-84897577-2f47-4ae3-9ca4-39b84ae06a98'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.884015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] The token cache is being invalidate because project 6bad303d7d484d9abbee20a19c3cb020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:24.884301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.884946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-84897577-2f47-4ae3-9ca4-39b84ae06a98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:24.886517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1198/2390] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:24 2022] DELETE /identity/v3/projects/6bad303d7d484d9abbee20a19c3cb020 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:24.891524 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:24.891813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:24.892088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:24.914669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:24.915899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:24.981245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1193/2391] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:24.990381 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:24.990681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:24.990951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:24.999105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:25.012025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:25.013249 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:25.021793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1199/2392] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:25.026015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.037027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae5df3-b405-4a8e-873b-1cc9551592c3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.041639 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.042617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.043517 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.043823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.044471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:25.044778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.046399 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.062026 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1194/2393] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:25.063681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.073384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99d062-7330-4e18-8d57-7a74ee1337f5 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.078993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.079599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.079902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.080187 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.080816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.082739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.164396 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1200/2394] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:25 2022] POST /identity/v3/projects => generated 381 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:25.169301 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.179327 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e32afb6-014b-419a-8011-d6c55924eb33 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.236277 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.236811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.237093 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.237431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.238066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.240038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.329597 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1195/2395] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:25 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:25.333718 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.344277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81523807-d1ec-496e-a3f1-d30c61527a06 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.348938 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.349677 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.349955 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.350228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.350897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.352480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.365252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1201/2396] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:25.367088 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.377669 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a6b4c9-f34e-4754-875e-6e245fa46768 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.382247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.382869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.383149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.383419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.384048 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.385628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf1b28e-bc57-47fe-b75a-a226f5e2abfe admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.395994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1196/2397] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:25.400051 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.410054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29e233-305b-4e69-8bf3-26468c7718d1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.414650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.415298 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.415580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.415858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] PATH_INFO: `/v3/projects/088f81a06fde4398b72f59b2249138df/users/34e8a4587c074b3c84fd8fe456b3cfea/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.430979 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] RBAC: Authorizing `identity:create_grant(project_id=088f81a06fde4398b72f59b2249138df, user_id=34e8a4587c074b3c84fd8fe456b3cfea, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.434433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.517371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1202/2398] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:25 2022] PUT /identity/v3/projects/088f81a06fde4398b72f59b2249138df/users/34e8a4587c074b3c84fd8fe456b3cfea/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:25.527505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.528959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.529240 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.557484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] MFA Rules not processed for user `34e8a4587c074b3c84fd8fe456b3cfea`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:25.559818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:25.601475 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:25.625310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1197/2399] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:25 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:25.630018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.691948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf09d5a-204e-4663-8d88-9809f371570c admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.696665 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.697264 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.697541 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.697810 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.698443 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.700501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.787485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1203/2400] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:25 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:25.791971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.803885 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa124b39-f21e-418f-a781-346ee6d9b14f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.859466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.860078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.860356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.860627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.861246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.863213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.929073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1198/2401] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:25 2022] POST /identity/v3/users => generated 296 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:25.933020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.943344 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3dba8-206a-41ed-9cc5-a040e5e849be admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.947977 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.948602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.949681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.949681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.950265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:25.951897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:25.962742 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1204/2402] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:25.967591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:25.977861 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18af661-187e-4518-a75f-52279e106fbb admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:25.983014 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:25.983678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:25.983960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:25.984235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] PATH_INFO: `/v3/projects/f0a5da26be274d4397689fba6e098b31/users/74a686731a9c46c18eb0efc97baf89c7/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:25.999607 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a5da26be274d4397689fba6e098b31, user_id=74a686731a9c46c18eb0efc97baf89c7, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:26.002246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:26.060178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1199/2403] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:25 2022] PUT /identity/v3/projects/f0a5da26be274d4397689fba6e098b31/users/74a686731a9c46c18eb0efc97baf89c7/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:26.064234 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:26.074925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5e025-927c-4f13-a63a-3738a61e24b1 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:26.130231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:26.130870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:26.131151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:26.131426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:26.132058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:26.133641 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba337d0-2bcb-4416-8e37-740db27156d2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:26.143663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1205/2404] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:26.147816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:26.158900 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd561d1-c65e-4d8c-b53c-92028b358f71 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:26.163538 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:26.164178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:26.164461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:26.164735 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] PATH_INFO: `/v3/projects/f0a5da26be274d4397689fba6e098b31/users/74a686731a9c46c18eb0efc97baf89c7/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:26.166707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a5da26be274d4397689fba6e098b31, user_id=74a686731a9c46c18eb0efc97baf89c7, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:26.169481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:26.179819 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1200/2405] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:26 2022] PUT /identity/v3/projects/f0a5da26be274d4397689fba6e098b31/users/74a686731a9c46c18eb0efc97baf89c7/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:26.190007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:26.190310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:26.190587 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:26.210386 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:26.219509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] MFA Rules not processed for user `74a686731a9c46c18eb0efc97baf89c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:26.220808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:26.288934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1206/2406] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:26 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:26.301439 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:26.303140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:26.360993 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f213e0-15f3-4717-8c88-d765c9ba223d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:26.364580 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:26.365168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:26.365448 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:26.365719 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:26.375510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:26.376930 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:26.386184 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1201/2407] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:26 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:27.220527 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:27.229737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3272147f-ac10-46e9-9ede-c3b9f819383e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:27.235358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a686731a9c46c18eb0efc97baf89c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0a5da26be274d4397689fba6e098b31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:27.236038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:27.236328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:27.236623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:27.237275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:27.237581 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:27.239252 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:27.251255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1207/2408] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:27.255628 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:27.265113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc964f3-2011-4976-94c6-22713483de7f service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:27.269585 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a686731a9c46c18eb0efc97baf89c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0a5da26be274d4397689fba6e098b31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:27.270178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:27.270460 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:27.270766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:27.271435 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:27.273352 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:27.351374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1202/2409] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:27 2022] POST /identity/v3/projects => generated 354 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:27.355957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:27.365987 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fec2d0-807d-4486-aff8-3f2898311c02 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:27.423042 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a686731a9c46c18eb0efc97baf89c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0a5da26be274d4397689fba6e098b31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:27.423647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:27.423932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:27.424209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:27.424852 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:27.425159 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:27.426780 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:27.438405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1208/2410] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:27.475862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:27.486696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3870a-b514-40aa-b85b-0b84edc745a5 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:27.491510 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74a686731a9c46c18eb0efc97baf89c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f0a5da26be274d4397689fba6e098b31', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:27.492147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:27.500992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:27.500992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] PATH_INFO: `/v3/projects/825a195599004429b20825273c52466e` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:27.500992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorizing `identity:delete_project(project_id=825a195599004429b20825273c52466e)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:27.500992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:27.508021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:27.608731 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '825a195599004429b20825273c52466e', 'request_id': 'req-a9e042d7-ca34-406c-92fb-02256f220287'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:27.694130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '825a195599004429b20825273c52466e', 'request_id': 'req-a9e042d7-ca34-406c-92fb-02256f220287'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:27.699962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '825a195599004429b20825273c52466e', 'request_id': 'req-a9e042d7-ca34-406c-92fb-02256f220287'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:27.711653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] The token cache is being invalidate because project 825a195599004429b20825273c52466e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:27.711935 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:27.712576 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a9e042d7-ca34-406c-92fb-02256f220287 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:27.714069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1203/2411] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:27 2022] DELETE /identity/v3/projects/825a195599004429b20825273c52466e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:28.275319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:28.278371 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:28.345204 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237206b-e455-4c0a-99cd-134ac1d98c76 tempest-VolumeQuotaClassesTest-883007396 tempest-VolumeQuotaClassesTest-883007396-project-admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:28.363959 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:28.404501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:28.405060 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:28.405338 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:28.405632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] PATH_INFO: `/v3/users/34e8a4587c074b3c84fd8fe456b3cfea` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:28.406835 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34e8a4587c074b3c84fd8fe456b3cfea)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:28.408754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:28.612103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34e8a4587c074b3c84fd8fe456b3cfea'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:28.624607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34e8a4587c074b3c84fd8fe456b3cfea'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:28.625297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34e8a4587c074b3c84fd8fe456b3cfea'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:28.633939 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34e8a4587c074b3c84fd8fe456b3cfea'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:28.635311 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34e8a4587c074b3c84fd8fe456b3cfea'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:28.727922 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1209/2412] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:28 2022] DELETE /identity/v3/users/34e8a4587c074b3c84fd8fe456b3cfea => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:28.732374 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:28.732876 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:28.733058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:28.753716 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:28.754973 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:28.815878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1204/2413] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:28.826158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:28.827585 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:28.906223 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732d4d97-c27f-48a3-a566-699167d31e8e admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:28.909889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:28.910503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:28.910811 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:28.911089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:28.921342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:28.922495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:28.940836 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1210/2414] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:29.364958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:29.366408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:29.379488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba72d3-d099-4cb8-a8de-7a7fe1cdd7e6 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:29.384189 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:29.384830 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:29.385203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:29.385479 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] PATH_INFO: `/v3/projects/088f81a06fde4398b72f59b2249138df` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:29.386600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=088f81a06fde4398b72f59b2249138df)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:29.388692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:29.501330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '088f81a06fde4398b72f59b2249138df', 'request_id': 'req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.567889 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '088f81a06fde4398b72f59b2249138df', 'request_id': 'req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.574063 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '088f81a06fde4398b72f59b2249138df', 'request_id': 'req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.585284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] The token cache is being invalidate because project 088f81a06fde4398b72f59b2249138df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:29.585570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.586195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.587722 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1205/2415] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:29 2022] DELETE /identity/v3/projects/088f81a06fde4398b72f59b2249138df => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:29.592862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:29.602329 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2acfdd-0db5-48c9-afe2-6e9a3c7fee4c service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:29.645618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:29.660829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:29.661463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:29.661740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:29.662011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] PATH_INFO: `/v3/users/74a686731a9c46c18eb0efc97baf89c7` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:29.663158 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a686731a9c46c18eb0efc97baf89c7)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:29.665152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:29.901996 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a686731a9c46c18eb0efc97baf89c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.914848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a686731a9c46c18eb0efc97baf89c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.915529 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a686731a9c46c18eb0efc97baf89c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.922814 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a686731a9c46c18eb0efc97baf89c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.923446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-042c41f5-bf20-4c03-95b5-0d8d9c7043dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a686731a9c46c18eb0efc97baf89c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:29.997839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1211/2416] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:29 2022] DELETE /identity/v3/users/74a686731a9c46c18eb0efc97baf89c7 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:30.393666 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:30.395156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:30.416032 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8339ef1-13c5-4d72-b1f0-a5cfbcd52b9e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:30.444317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:30.483550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:30.484187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:30.484465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:30.484736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] PATH_INFO: `/v3/projects/f0a5da26be274d4397689fba6e098b31` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:30.485853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a5da26be274d4397689fba6e098b31)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:30.487837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:30.665367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a5da26be274d4397689fba6e098b31', 'request_id': 'req-8d2ffa7e-d265-4a0b-8441-eae81d01054a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:30.751155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a5da26be274d4397689fba6e098b31', 'request_id': 'req-8d2ffa7e-d265-4a0b-8441-eae81d01054a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:30.757096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a5da26be274d4397689fba6e098b31', 'request_id': 'req-8d2ffa7e-d265-4a0b-8441-eae81d01054a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:30.768989 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] The token cache is being invalidate because project f0a5da26be274d4397689fba6e098b31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:30.769275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:30.769916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-8d2ffa7e-d265-4a0b-8441-eae81d01054a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:30.771454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1206/2417] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:30 2022] DELETE /identity/v3/projects/f0a5da26be274d4397689fba6e098b31 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:30.777954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:30.778719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:30.779003 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:30.799706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:30.800889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:30.870786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1212/2418] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:30.879795 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:30.880097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:30.882364 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:30.900237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:30.901443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:30.910516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1207/2419] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:30.914916 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:30.925574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c02ee-db98-4071-8e82-ff407d4a6dcc None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:30.930272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:30.930929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:30.931213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:30.931483 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:30.932129 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:30.932429 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:30.934290 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:30.946152 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1213/2420] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:30.955215 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:30.964893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61bfc-c60f-4cc9-afd7-d6bd6d7bdad2 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:30.970253 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:30.970894 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:30.971175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:30.971449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:30.972098 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:30.974057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.065574 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1208/2421] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:30 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:31.070143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:31.080436 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac07a952-2367-4d94-9cf7-3464a6c3618f admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:31.129203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:31.138863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:31.139464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.139740 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.140011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.140661 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:31.142557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.221303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1214/2422] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:31 2022] POST /identity/v3/users => generated 287 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:31.230891 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:31.241686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80013ecd-a845-4887-b6d7-a523fa4cb1f5 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:31.247072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:31.249511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.249511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.249511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.249511 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:31.250616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.269401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1209/2423] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:31.269517 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:31.276880 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29167dda-352a-48cf-912e-8fe0cd61e631 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:31.282275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:31.282991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.283445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.283792 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.284505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:31.286155 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5aca1d-bdbe-4380-84b4-4e2acf9a9a25 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.298243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1215/2424] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:31.305157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:31.312685 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab300-e2c4-4226-bb01-114cbdb93970 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:31.317829 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:31.318488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.318797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.319072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] PATH_INFO: `/v3/projects/3a5ca098647949a89a1d3d96ed6e5ed4/users/b9d4cbdf1de0440a934d97916734338e/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.340551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5ca098647949a89a1d3d96ed6e5ed4, user_id=b9d4cbdf1de0440a934d97916734338e, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:31.343970 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.420540 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1210/2425] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:31 2022] PUT /identity/v3/projects/3a5ca098647949a89a1d3d96ed6e5ed4/users/b9d4cbdf1de0440a934d97916734338e/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:31.431495 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.431774 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.432043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.462110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] MFA Rules not processed for user `b9d4cbdf1de0440a934d97916734338e`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:31.463372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:31.529929 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1216/2426] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:31.540461 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:31.542138 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:31.598017 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:31.601763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aee81c-aba0-43f1-a113-391de99cffd9 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:31.605493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:31.606123 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:31.606403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:31.606805 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:31.615150 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:31.616612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a4c67-6e30-4100-bbcf-25454252ae79 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:31.625995 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1211/2427] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:34.832885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:34.834287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:34.898114 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb49ff9-2aca-4408-8453-1c2a98d24293 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:34.902622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:34.903397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:34.903625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:34.903899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] PATH_INFO: `/v3/users/c642f9d09ca94ec0b9d3110ace798456` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:34.905015 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c642f9d09ca94ec0b9d3110ace798456)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:34.906957 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:35.136919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c642f9d09ca94ec0b9d3110ace798456'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:35.148636 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:35.151319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c642f9d09ca94ec0b9d3110ace798456'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:35.152066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c642f9d09ca94ec0b9d3110ace798456'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:35.160111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c642f9d09ca94ec0b9d3110ace798456'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:35.161020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c642f9d09ca94ec0b9d3110ace798456'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:35.232229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1217/2428] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:34 2022] DELETE /identity/v3/users/c642f9d09ca94ec0b9d3110ace798456 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:35.236988 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4bb8-2f52-4a9f-bac4-77969ff8f96f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:35.237478 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4bb8-2f52-4a9f-bac4-77969ff8f96f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:35.237695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4bb8-2f52-4a9f-bac4-77969ff8f96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:35.257092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-202d4bb8-2f52-4a9f-bac4-77969ff8f96f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:35.258291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-202d4bb8-2f52-4a9f-bac4-77969ff8f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:35.317730 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1212/2429] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:35.327919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:35.329369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:35.411533 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62407a44-0a30-49d2-8cf7-78c6567b8083 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:35.415219 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:35.415861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:35.416136 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:35.416403 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:35.426625 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:35.428089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:35.437734 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1218/2430] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:35.444791 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223765-a318-4446-973f-d9a5a5900805 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:35.449516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223765-a318-4446-973f-d9a5a5900805 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:35.449516 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de223765-a318-4446-973f-d9a5a5900805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:35.467120 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-de223765-a318-4446-973f-d9a5a5900805 None None] MFA Rules not processed for user `b9d4cbdf1de0440a934d97916734338e`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:35.468341 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-de223765-a318-4446-973f-d9a5a5900805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:35.526611 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1213/2431] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:36.067936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.069365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:36.082410 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5122776-3517-405a-9274-2ef37b80ab97 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.087902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.088538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.090028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.090028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] PATH_INFO: `/v3/projects/6c5b34f2026d469f85c774cf9012d0bf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.090972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c5b34f2026d469f85c774cf9012d0bf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.092934 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.187744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c5b34f2026d469f85c774cf9012d0bf', 'request_id': 'req-1d450cd4-d089-4764-a6ef-de7375515f45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:36.236754 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c5b34f2026d469f85c774cf9012d0bf', 'request_id': 'req-1d450cd4-d089-4764-a6ef-de7375515f45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:36.243696 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c5b34f2026d469f85c774cf9012d0bf', 'request_id': 'req-1d450cd4-d089-4764-a6ef-de7375515f45'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:36.254700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] The token cache is being invalidate because project 6c5b34f2026d469f85c774cf9012d0bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:36.254993 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:36.255632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1d450cd4-d089-4764-a6ef-de7375515f45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:36.257131 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1219/2432] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:36 2022] DELETE /identity/v3/projects/6c5b34f2026d469f85c774cf9012d0bf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:36.262154 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.262879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.262879 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.282851 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:36.284053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:36.346905 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1214/2433] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:36.354380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.354688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.354963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.377330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:36.378531 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:36.386405 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1220/2434] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:36.397300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.401619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68c251-3dde-4b7e-967b-d923cdfd110d None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.407673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.408279 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.408560 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.408860 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.409630 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:36.410058 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.411706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.424382 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1215/2435] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:36.428691 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.437933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fca5de-97ad-43bb-9f61-bebe28ce7198 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.442879 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.443524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.443809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.444128 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.444796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.446862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.536947 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1221/2436] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:36 2022] POST /identity/v3/projects => generated 359 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:36.540504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.550987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba9ee3-6c15-4ecc-be8f-ee0dc52342ff admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.571481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=100 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:36.609779 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.610389 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.610783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.610999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.611589 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.613552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.708497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1216/2437] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:36 2022] POST /identity/v3/users => generated 280 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:36.712724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.725601 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0010394b-df61-47e5-8214-4cf07a1373d4 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.730509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.731907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.732203 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.732488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.733104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.734707 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.744761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1222/2438] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:36.749314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.759797 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03601031-0884-496f-9ce8-097ffc3449b3 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.765342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.769300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.769300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.769300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.769300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.769300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78fd9d2-aeb7-4844-82b2-fc6a051fc1a2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.778967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1217/2439] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:36.783125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:36.789409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:36.793240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab5a80-bd36-4284-a657-ee811929b751 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:36.799194 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:36.799889 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.800164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.800433 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] PATH_INFO: `/v3/projects/fc534288b78a42b9b7b0fc0e25a5eb15/users/8d396ae3a4d042eb9c44742b16b1f9c9/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.816265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc534288b78a42b9b7b0fc0e25a5eb15, user_id=8d396ae3a4d042eb9c44742b16b1f9c9, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:36.819027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:36.883027 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1223/2440] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:36 2022] PUT /identity/v3/projects/fc534288b78a42b9b7b0fc0e25a5eb15/users/8d396ae3a4d042eb9c44742b16b1f9c9/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:36.895174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:36.895456 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:36.895815 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:36.937760 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] MFA Rules not processed for user `8d396ae3a4d042eb9c44742b16b1f9c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:36.939074 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:37.026655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1218/2441] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:36 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:37.062415 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:37.064154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:37.127348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29d132-2406-4741-bc9b-57290085d977 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:37.132956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:37.133574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:37.133860 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:37.134132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:37.143319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:37.144768 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dbb5ba-f9dd-4a97-bee9-2c74acb5cbb5 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:37.155827 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1224/2442] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:37.177817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:37.282723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa12da9-d606-4264-8095-5e22b754b7be None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:37.288295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:37.288954 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:37.289235 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:37.289506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] PATH_INFO: `/v3/users/8d396ae3a4d042eb9c44742b16b1f9c9` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:37.291193 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d396ae3a4d042eb9c44742b16b1f9c9)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:37.293216 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:37.439250 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:37.455457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d396ae3a4d042eb9c44742b16b1f9c9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:37.456174 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d396ae3a4d042eb9c44742b16b1f9c9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:37.470021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d396ae3a4d042eb9c44742b16b1f9c9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:37.511375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d396ae3a4d042eb9c44742b16b1f9c9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:37.512182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d396ae3a4d042eb9c44742b16b1f9c9'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:37.528452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1219/2443] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:37 2022] DELETE /identity/v3/users/8d396ae3a4d042eb9c44742b16b1f9c9 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:37.534575 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:37.534888 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:37.535165 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:37.555825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:37.561991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:37.619686 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1225/2444] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:37.630837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:37.638573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:37.708180 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26076fe-2e84-4aa6-bfeb-ad796c8865e4 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:37.712084 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:37.712709 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:37.712992 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:37.713289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:37.723896 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:37.725339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bb7340-310a-4f88-a095-38d840748cdf service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:37.738914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1220/2445] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:38.182085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:38.183554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:38.198098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c4e130-a881-4f5a-ade1-57e1373d24fe None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:38.203572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:38.204218 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.205230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.205882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] PATH_INFO: `/v3/projects/fc534288b78a42b9b7b0fc0e25a5eb15` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.207100 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc534288b78a42b9b7b0fc0e25a5eb15)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:38.209096 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:38.541522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc534288b78a42b9b7b0fc0e25a5eb15', 'request_id': 'req-ff720e39-4be4-4a53-9642-44a814fd5e90'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:38.625247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc534288b78a42b9b7b0fc0e25a5eb15', 'request_id': 'req-ff720e39-4be4-4a53-9642-44a814fd5e90'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:38.631980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc534288b78a42b9b7b0fc0e25a5eb15', 'request_id': 'req-ff720e39-4be4-4a53-9642-44a814fd5e90'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:38.644006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] The token cache is being invalidate because project fc534288b78a42b9b7b0fc0e25a5eb15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:38.644293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:38.644952 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-ff720e39-4be4-4a53-9642-44a814fd5e90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:38.646427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1226/2446] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:38 2022] DELETE /identity/v3/projects/fc534288b78a42b9b7b0fc0e25a5eb15 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:38.651302 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.651813 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.651927 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.671401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:38.672904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:38.732661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1221/2447] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:38.741348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.741632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.741908 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.757978 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:38.763766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:38.765228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:38.773265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1227/2448] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:38.777636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:38.787802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531312d5-37a1-47d2-b365-e557aa9e446f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:38.792594 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:38.793132 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.793908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.794168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.794842 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:38.795147 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:38.796736 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:38.808659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1222/2449] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:38.813635 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:38.823196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da4830-41c7-4ca5-b52b-51e278cebd23 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:38.827846 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:38.833119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.833119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.833119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.833119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:38.833119 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:38.912208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1228/2450] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:38 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:38.916447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:38.927523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83e2566-fc0e-4a95-8f50-b202a7069285 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:38.968977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:38.986602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:38.987243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:38.987525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:38.987801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:38.988573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:38.990481 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:39.071449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1223/2451] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:38 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:39.075769 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:39.086150 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b2ba3-0580-4957-9940-6111e67235e2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:39.091545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:39.092143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:39.092422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:39.092692 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:39.093326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:39.095009 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:39.105020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1229/2452] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:39.109801 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:39.119956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed47179-e3c6-4967-9d23-a45d3abc0d08 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:39.124723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:39.125365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:39.125646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:39.125917 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:39.126549 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:39.128164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6956bb-eec7-4fe6-8eeb-9dab9a4c465c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:39.138794 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1224/2453] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:39.142784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:39.153342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127cb0d-e2e9-4b2c-986c-fe5014b870e9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:39.158160 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:39.158932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:39.159196 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:39.159481 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] PATH_INFO: `/v3/projects/f96822286c4c4c9d923ca5d1469cc53f/users/7be5554b34004559a4d4a67027a504eb/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:39.175048 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96822286c4c4c9d923ca5d1469cc53f, user_id=7be5554b34004559a4d4a67027a504eb, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:39.177690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:39.248547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1230/2454] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:39 2022] PUT /identity/v3/projects/f96822286c4c4c9d923ca5d1469cc53f/users/7be5554b34004559a4d4a67027a504eb/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:39.313211 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:39.313497 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:39.313777 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:39.342126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] MFA Rules not processed for user `7be5554b34004559a4d4a67027a504eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:39.343296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:39.408748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1225/2455] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:39.420339 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:39.422962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:39.504758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b97b5e-0067-441e-97cc-e5cccd635791 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:39.508501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:39.509165 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:39.509442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:39.509713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:39.518377 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:39.519820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3a0338-9e74-41ed-b75f-b51553355050 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:39.528705 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1231/2456] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:43.495408 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:43.557818 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72275ae-6c9a-44e5-b815-f0501e408d02 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:43.563299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:43.563936 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:43.565446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:43.565446 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] PATH_INFO: `/v3/users/7be5554b34004559a4d4a67027a504eb` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:43.565663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be5554b34004559a4d4a67027a504eb)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:43.567601 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:43.776245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be5554b34004559a4d4a67027a504eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:43.776898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be5554b34004559a4d4a67027a504eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:43.781921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:43.789706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be5554b34004559a4d4a67027a504eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:43.859920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be5554b34004559a4d4a67027a504eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:43.860648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be5554b34004559a4d4a67027a504eb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:43.873646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1226/2457] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:43 2022] DELETE /identity/v3/users/7be5554b34004559a4d4a67027a504eb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:43.878176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:43.878702 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:43.878824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:43.898419 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:43.902861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:43.958192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1232/2458] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:43.968206 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:43.969771 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:44.036143 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f3159-f1c9-49a3-b003-2c42d3ec1975 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:44.040028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:44.040661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.040933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.041208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.052328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:44.053766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e3262-4689-4bc9-8690-95097d5cad02 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:44.065622 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1227/2459] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:44.512596 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:44.521089 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:44.526945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f09e233-0258-4ca7-80f9-f0c6f13985b3 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:44.532878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:44.533520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.533795 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.534063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] PATH_INFO: `/v3/projects/f96822286c4c4c9d923ca5d1469cc53f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.535211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96822286c4c4c9d923ca5d1469cc53f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:44.537104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:44.561948 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:44.685052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96822286c4c4c9d923ca5d1469cc53f', 'request_id': 'req-1799343f-95f4-4e05-983d-4e6623d0ecdd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:44.768349 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96822286c4c4c9d923ca5d1469cc53f', 'request_id': 'req-1799343f-95f4-4e05-983d-4e6623d0ecdd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:44.775231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96822286c4c4c9d923ca5d1469cc53f', 'request_id': 'req-1799343f-95f4-4e05-983d-4e6623d0ecdd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:44.786326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] The token cache is being invalidate because project f96822286c4c4c9d923ca5d1469cc53f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:44.790882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:44.790882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1799343f-95f4-4e05-983d-4e6623d0ecdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:44.790882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1233/2460] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:44 2022] DELETE /identity/v3/projects/f96822286c4c4c9d923ca5d1469cc53f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:44.793705 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.794157 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.794314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.814361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:44.815598 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:44.876152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1228/2461] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:44.883853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.884362 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.884473 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.905189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:44.906402 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:44.914316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1234/2462] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:44.918452 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:44.928778 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2713146-4bfd-4cd6-8175-860b47ff239f None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:44.934241 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:44.934929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.935339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.935817 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.936652 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:44.936959 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:44.938600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:44.951067 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1229/2463] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:12:44.955586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:44.964854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265abbd-343d-4924-a471-3b86db9a4b30 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:44.970220 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:44.970853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:44.971132 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:44.971404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:44.972189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:44.974075 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.037894 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1235/2464] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:44 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:45.041972 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.052839 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced557f-4aaa-4135-988f-7ec08a5b8bf2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.109370 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.109971 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.110247 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.110518 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.111182 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.113083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.121024 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:45.194784 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1230/2465] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:45 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:45.195994 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.206539 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03034d8a-e0db-4e90-a86b-b43e72ee2062 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.212106 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.212897 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.213248 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.213655 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.214525 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.216008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.218720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.225311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:45.229984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1236/2466] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:45.231108 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.242273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cef1519-9535-41e4-bcc0-ea10b79ec009 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.249560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.250310 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.250789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.251172 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.251913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.253699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.265098 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1237/2467] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:45.269504 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.282151 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a955a712-f922-40df-896e-cdb4d6b7606b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.287622 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.288364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.288579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.288849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] PATH_INFO: `/v3/projects/257edeea1ce9438cab81087e5ab4297b/users/5c5ccbeaa0814badb0e5956c27ee4c23/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.301541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020024-2ed0-4c4f-b45c-e40340dc382c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.306181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.310604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.310604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.310604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.311093 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=257edeea1ce9438cab81087e5ab4297b, user_id=5c5ccbeaa0814badb0e5956c27ee4c23, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.311198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:45.311498 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2002ac3-162b-44cd-9eb7-34553f3a10f6 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.381713 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.383205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.387538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1238/2468] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:45 2022] PUT /identity/v3/projects/257edeea1ce9438cab81087e5ab4297b/users/5c5ccbeaa0814badb0e5956c27ee4c23/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:45.395130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1231/2469] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:45.399372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.399679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.399950 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.430989 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] MFA Rules not processed for user `5c5ccbeaa0814badb0e5956c27ee4c23`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:45.432283 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:45.499354 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1239/2470] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:45.503617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.515347 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a7f9d-d373-4461-b82f-64613496b45d service cinder] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.571401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.571921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.572246 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.572494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.573133 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.575043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.678857 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1232/2471] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:45.683006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.695041 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905e47d-0793-4de5-a1e8-adb937e6b19b None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.715176 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:45.752394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.753225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.753442 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.753658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.754208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.756277 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.837372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1240/2472] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:45 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:45.841126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.853372 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ddad9-2884-4e08-92b5-65ef458b5389 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.859668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.860291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.860670 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.860854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.861853 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.863523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.875933 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1233/2473] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 21 15:12:45.881564 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.892391 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf60e11-fbb7-4b11-a371-ee4a76dd7f91 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.897028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.897626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.897907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.898180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.898933 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.900514 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7671c-3691-4a73-a04a-387ffa9738a2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.913711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1241/2474] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:45.914893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:45.926411 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a37328-6c2c-4f00-bb66-8b68b0aabc73 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:45.931485 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:45.932134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:45.932415 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:45.932821 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] PATH_INFO: `/v3/projects/4e17c0f6d7a047e68f78b8aac3ee06af/users/09e2c569c80b46bfb192bb9c796a062c/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:45.948161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e17c0f6d7a047e68f78b8aac3ee06af, user_id=09e2c569c80b46bfb192bb9c796a062c, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:45.952049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:45.954380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:46.049881 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1234/2475] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:45 2022] PUT /identity/v3/projects/4e17c0f6d7a047e68f78b8aac3ee06af/users/09e2c569c80b46bfb192bb9c796a062c/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:46.061085 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.061340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.061620 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.090499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] MFA Rules not processed for user `09e2c569c80b46bfb192bb9c796a062c`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:46.091695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:46.157081 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1242/2476] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:46.161673 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.224016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a7538-a89c-481d-aaa1-51a51b2336f2 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.228710 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.229305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.229649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.229928 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.230555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.232490 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.341040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1235/2477] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:46 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:46.345278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.355926 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773fdad9-3697-47f0-b9a7-06b9d939b3ca None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.413241 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.413861 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.414137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.414414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.415156 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.417073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.487182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1243/2478] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:46 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:46.488421 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.499596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7619cb-89a7-4938-a360-f9058d3a579f admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.504268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.504880 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.505155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.505424 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.506053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.507676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.518403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1236/2479] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:46.523065 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.534304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e2696-a60a-4670-88e4-4a5db0409f97 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.538924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.539580 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.539858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.540130 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] PATH_INFO: `/v3/projects/07fef7178ab848179b027939e495de68/users/57436f95424a45aea99cfc003dff0d50/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.557670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fef7178ab848179b027939e495de68, user_id=57436f95424a45aea99cfc003dff0d50, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.560347 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.637301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:46.643556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1244/2480] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:46 2022] PUT /identity/v3/projects/07fef7178ab848179b027939e495de68/users/57436f95424a45aea99cfc003dff0d50/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:46.651092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.657696 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce90ea46-6874-4afa-bb19-4c09a960d494 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.713222 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.713816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.714090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.714357 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.715008 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.716615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1993d-8096-48d4-9646-b376dce76db6 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.726453 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1237/2481] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:46.730653 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.740807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca10c9a-d145-46cb-9020-47873a8f7190 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.745501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.746146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.746425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.746739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] PATH_INFO: `/v3/projects/07fef7178ab848179b027939e495de68/users/57436f95424a45aea99cfc003dff0d50/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.749170 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07fef7178ab848179b027939e495de68, user_id=57436f95424a45aea99cfc003dff0d50, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:46.751864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:46.761799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1245/2482] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:46 2022] PUT /identity/v3/projects/07fef7178ab848179b027939e495de68/users/57436f95424a45aea99cfc003dff0d50/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:46.772315 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.772600 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.772869 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:46.801506 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] MFA Rules not processed for user `57436f95424a45aea99cfc003dff0d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:46.802776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:46.853168 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:46.871503 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1238/2483] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:46 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:46.901783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:46.962316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b343b2-69b4-45d6-b653-76acf3d1699d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:46.966229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:46.966872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:46.967611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:46.967611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:47.027368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:47.031913 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:47.038012 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1246/2484] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:50.058509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:50.071185 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d9d27-61c4-478b-b125-0f40422902d9 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:50.075156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:50.075787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:50.076057 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:50.076324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:50.146908 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:50.148644 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51073f5b-a8fd-43ff-b14a-51d947866ed3 service cinder] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:50.160313 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1239/2485] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:51.652425 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:51.654369 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:51.666932 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f459b35f-272d-4768-bfef-4d29a089f765 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:51.670966 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:51.671817 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:51.672353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:51.672793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:51.682958 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:51.684389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:51.695016 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1247/2486] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:53.969181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540cbe-46e0-4918-a67a-e9d22ada2398 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:53.969181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540cbe-46e0-4918-a67a-e9d22ada2398 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:53.969181 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf540cbe-46e0-4918-a67a-e9d22ada2398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:53.983122 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-bf540cbe-46e0-4918-a67a-e9d22ada2398 None None] MFA Rules not processed for user `f71ffc3d05d4470fa533cfd6f22184b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:53.984378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-bf540cbe-46e0-4918-a67a-e9d22ada2398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:54.044473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1240/2487] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:54.902509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:54.903980 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:54.978753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53d21fa-b188-4d6e-bb65-0d6cb63cdb12 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:54.983463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:54.984099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:54.984375 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:54.984984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] PATH_INFO: `/v3/users/a9045eaeb805432da2112031ad92c74f` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:54.986117 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9045eaeb805432da2112031ad92c74f)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:54.988962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:55.072690 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:55.188247 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9045eaeb805432da2112031ad92c74f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:55.206468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9045eaeb805432da2112031ad92c74f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:55.206468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9045eaeb805432da2112031ad92c74f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:55.210501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9045eaeb805432da2112031ad92c74f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:55.211229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9045eaeb805432da2112031ad92c74f'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:55.305763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1248/2488] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:54 2022] DELETE /identity/v3/users/a9045eaeb805432da2112031ad92c74f => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:55.310228 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:55.310765 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:55.310912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:55.331513 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:55.332738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:55.394482 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1241/2489] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:55.404582 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:55.407057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:55.468557 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc10aa76-0c5f-4ac4-aa98-0fd7f19426bb admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:55.472485 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:55.473113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:55.473604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:55.473885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:55.483874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:55.485316 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:55.495739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1249/2490] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:12:55.948190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:55.954810 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:55.967417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeaeea4-57df-489d-9813-902fe9e5957c None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:55.972562 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:55.973234 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:55.973552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:55.973763 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] PATH_INFO: `/v3/projects/80378bd364cf43018a211197fdf30ee4` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:55.980619 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80378bd364cf43018a211197fdf30ee4)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:55.989956 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:56.091521 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:56.110190 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80378bd364cf43018a211197fdf30ee4', 'request_id': 'req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.208191 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80378bd364cf43018a211197fdf30ee4', 'request_id': 'req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.218856 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80378bd364cf43018a211197fdf30ee4', 'request_id': 'req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.233787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] The token cache is being invalidate because project 80378bd364cf43018a211197fdf30ee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:56.234161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.235078 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.237672 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1242/2491] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:55 2022] DELETE /identity/v3/projects/80378bd364cf43018a211197fdf30ee4 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:56.243592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:56.256418 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ac393-2d05-43fb-974c-7c7a7c67ca1d service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:56.367315 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:56.368384 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:56.368863 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:56.369273 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] PATH_INFO: `/v3/users/abb44e97e37d406297f13b9e34734dfd` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:56.370670 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb44e97e37d406297f13b9e34734dfd)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:56.373572 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:56.617261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb44e97e37d406297f13b9e34734dfd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.619304 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:56.630005 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb44e97e37d406297f13b9e34734dfd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.630722 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb44e97e37d406297f13b9e34734dfd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.637555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb44e97e37d406297f13b9e34734dfd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.638180 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-0fc313e1-874c-4b20-ac3c-9d6aa499a2be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb44e97e37d406297f13b9e34734dfd'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:56.753133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1250/2492] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:56 2022] DELETE /identity/v3/users/abb44e97e37d406297f13b9e34734dfd => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:57.169476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.171042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.185443 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c86b98-f9dd-4c05-ab03-39cfe5fc5c46 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.243911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.244551 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.244834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.245115 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] PATH_INFO: `/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.246243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6a3b4c768ec42db83d678d9fa3c1594)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.248289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.351297 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6a3b4c768ec42db83d678d9fa3c1594', 'request_id': 'req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.426086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6a3b4c768ec42db83d678d9fa3c1594', 'request_id': 'req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.432552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6a3b4c768ec42db83d678d9fa3c1594', 'request_id': 'req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.447957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] The token cache is being invalidate because project a6a3b4c768ec42db83d678d9fa3c1594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:57.448256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.449130 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.450811 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1243/2493] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:57 2022] DELETE /identity/v3/projects/a6a3b4c768ec42db83d678d9fa3c1594 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:57.456153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba725cd-e624-409f-8de4-33d09bcf4f12 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.456632 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba725cd-e624-409f-8de4-33d09bcf4f12 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.456747 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba725cd-e624-409f-8de4-33d09bcf4f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.480357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-aba725cd-e624-409f-8de4-33d09bcf4f12 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:57.481676 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-aba725cd-e624-409f-8de4-33d09bcf4f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.520101 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.521520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.543179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1251/2494] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:57.554272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.554560 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.554870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.572823 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:57.575416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:57.576615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.584674 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1252/2495] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:57.588711 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.588848 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dca56e4-d92c-449d-9ff6-d50a6c72bc23 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.593677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.594317 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.594599 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.594904 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] PATH_INFO: `/v3/users/5c5ccbeaa0814badb0e5956c27ee4c23` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.596354 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5ccbeaa0814badb0e5956c27ee4c23)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.597882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c23ada4-4e92-45ac-a561-5c9bd80532c7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.598252 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.602719 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.603251 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.603520 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.603789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.604446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:12:57.604746 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.606301 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.619284 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1253/2496] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 21 15:12:57.628628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.638438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c78a1c-31f9-4c56-9fae-1a609a6d7b1d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.643245 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.643797 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.644080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.644359 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.644975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.646867 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.729874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1254/2497] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:57 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:57.733796 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.760309 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5ccbeaa0814badb0e5956c27ee4c23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.761075 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5ccbeaa0814badb0e5956c27ee4c23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.774249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5ccbeaa0814badb0e5956c27ee4c23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.800278 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc517b-5062-4aaa-8b78-63e86fd17cdf admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.805184 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.805825 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.806099 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.806368 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.807030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.808981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.825245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5ccbeaa0814badb0e5956c27ee4c23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.825870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-a6816e74-50e1-4387-b020-b35487aa85ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5ccbeaa0814badb0e5956c27ee4c23'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:57.839575 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1244/2498] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:57 2022] DELETE /identity/v3/users/5c5ccbeaa0814badb0e5956c27ee4c23 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:57.843854 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.844348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.844783 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.867556 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:57.869149 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.881105 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1255/2499] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:57 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:57.881915 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.883561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.895964 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb73e7-afa8-4f29-897d-8716c967d62d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:57.907878 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:57.934541 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1245/2500] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:57.944203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:57.947586 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:57.962942 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:57.964148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:57.964497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:57.964839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:57.965808 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:57.969044 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:57.990608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1256/2501] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:57.991873 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.004422 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bcefd2-e09a-44ce-95f0-c27787367577 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.010342 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.011066 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.011414 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.011793 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.012460 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.014116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.024784 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1257/2502] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:58.025646 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5268dee6-9bf0-42a8-a503-cd09a1c12a95 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.028907 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.031831 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.032582 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.032867 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.033660 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.040549 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4eea57-6557-4a3a-84b6-22dff742d332 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.044700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.045669 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.046140 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0017-429c-441a-b8f7-daed507a42cd service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.046446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.046822 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.047409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] PATH_INFO: `/v3/projects/37ca014da4334317afdfadf95250f857/users/3cbb457fed844cfd99b664e694cefccb/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.057243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1246/2503] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:12:58.064018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ca014da4334317afdfadf95250f857, user_id=3cbb457fed844cfd99b664e694cefccb, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.066826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.159935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1258/2504] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:58 2022] PUT /identity/v3/projects/37ca014da4334317afdfadf95250f857/users/3cbb457fed844cfd99b664e694cefccb/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:58.170365 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.170912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.171561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.192367 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:58.201013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] MFA Rules not processed for user `3cbb457fed844cfd99b664e694cefccb`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:58.202257 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:58.271505 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1247/2505] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:12:58.274112 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.348039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2518a65a-cdfb-4bf4-8cdd-4590bc3af21b admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.352720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.353322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.353599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.353864 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.354493 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.356411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.450563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1259/2506] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:58 2022] POST /identity/v3/projects => generated 387 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:58.454434 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.467044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b2c53-5b8b-4f0e-861d-e85a7ec90e6b None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.544320 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.544945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.545221 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.545494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.546164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.548390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.639759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1248/2507] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:58 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:58.643876 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.659045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9acfb8-77ed-4da0-9f37-72bd94830b05 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.664025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.664724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.665006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.665279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.665924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.667554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.683756 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1260/2508] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:58.687762 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.709249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307d940-8449-4061-a13a-d961f5112b04 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.714159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.714845 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.715245 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.715526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.716161 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.717803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.731824 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1249/2509] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:58.733358 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.744118 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:58.746087 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:58.748195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc674da-5a8d-4095-8b39-07070dbe45f4 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.753366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.754231 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.754438 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.754813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] PATH_INFO: `/v3/projects/1ef9cb50a1e1489ca3a787883fea95d6/users/7911c5df656144a1a87772532ba70119/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.761994 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03c92f-f641-4d30-b88e-7181309f2502 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:58.767314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:58.768938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.769281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.769616 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] PATH_INFO: `/v3/projects/257edeea1ce9438cab81087e5ab4297b` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.771507 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=257edeea1ce9438cab81087e5ab4297b)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.772965 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.773191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ef9cb50a1e1489ca3a787883fea95d6, user_id=7911c5df656144a1a87772532ba70119, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:58.776975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbd7a82-7ae7-4462-9f04-c463ef9e069c admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:58.867028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1261/2510] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:58 2022] PUT /identity/v3/projects/1ef9cb50a1e1489ca3a787883fea95d6/users/7911c5df656144a1a87772532ba70119/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:58.877532 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:58.878110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:58.878553 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:58.908072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '257edeea1ce9438cab81087e5ab4297b', 'request_id': 'req-2c07c203-78ed-4b08-bf35-8ab15315cd4d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:58.924623 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] MFA Rules not processed for user `7911c5df656144a1a87772532ba70119`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:12:58.925289 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:58.939807 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:58.969041 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '257edeea1ce9438cab81087e5ab4297b', 'request_id': 'req-2c07c203-78ed-4b08-bf35-8ab15315cd4d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:58.979520 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '257edeea1ce9438cab81087e5ab4297b', 'request_id': 'req-2c07c203-78ed-4b08-bf35-8ab15315cd4d'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:58.989493 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:58.993997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] The token cache is being invalidate because project 257edeea1ce9438cab81087e5ab4297b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:12:58.994398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:58.995248 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:58.997007 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1250/2511] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:12:58 2022] DELETE /identity/v3/projects/257edeea1ce9438cab81087e5ab4297b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:59.001337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.003325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:59.023017 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1262/2512] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:12:59.027148 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.028492 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:59.100103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c07c203-78ed-4b08-bf35-8ab15315cd4d admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.107117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.107877 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.108264 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.108748 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] PATH_INFO: `/v3/users/09e2c569c80b46bfb192bb9c796a062c` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.110655 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e2c569c80b46bfb192bb9c796a062c)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.113080 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.126743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5436288-fce6-4c5c-886f-b2f8429321c7 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.134268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.135122 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.135450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.135833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.136535 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.138679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.246032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1263/2513] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:12:59 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:12:59.249693 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.341945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadcb705-1011-4e0e-b986-90c1dff28395 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.348146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.348758 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.349111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.349699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.352026 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.353997 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.366755 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e2c569c80b46bfb192bb9c796a062c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:59.367530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e2c569c80b46bfb192bb9c796a062c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:59.381155 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e2c569c80b46bfb192bb9c796a062c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:59.465718 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e2c569c80b46bfb192bb9c796a062c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:59.466325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e2c569c80b46bfb192bb9c796a062c'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:12:59.473484 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1264/2514] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:12:59 2022] POST /identity/v3/users => generated 300 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:12:59.477586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.479593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:12:59.490251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1251/2515] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:12:59 2022] DELETE /identity/v3/users/09e2c569c80b46bfb192bb9c796a062c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:12:59.493038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1724c67-2efb-43e5-baee-514c0b668268 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.529537 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:12:59.569054 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.569731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.570227 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.570534 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.571505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.573232 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.585127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1265/2516] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:59.589019 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.601785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251edce3-6882-4bda-a5fe-cfcd6693328a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.609766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.610793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.611788 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.612664 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] PATH_INFO: `/v3/projects/2f78cf71665d4fc5be5038d488422721/users/ee98c450ea03478a9a11d3429ffe143a/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.633438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f78cf71665d4fc5be5038d488422721, user_id=ee98c450ea03478a9a11d3429ffe143a, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.637444 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.833232 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1252/2517] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:59 2022] PUT /identity/v3/projects/2f78cf71665d4fc5be5038d488422721/users/ee98c450ea03478a9a11d3429ffe143a/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:12:59.837556 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.852723 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c4241-c611-469c-960c-426bcb2249d0 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:12:59.931891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:12:59.932843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:12:59.933854 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:12:59.934146 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:12:59.934886 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:12:59.936600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550b670-5f69-4665-95e6-da818c2499e4 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:12:59.969372 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1266/2518] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:12:59.974310 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:12:59.995201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de85bc-9055-462c-8357-9a3b7b9f6c28 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:00.001390 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:00.002570 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:00.004787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:00.004787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] PATH_INFO: `/v3/projects/2f78cf71665d4fc5be5038d488422721/users/ee98c450ea03478a9a11d3429ffe143a/roles/55a557362b9542629b4209da47492ebf` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:00.006418 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f78cf71665d4fc5be5038d488422721, user_id=ee98c450ea03478a9a11d3429ffe143a, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:00.009463 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:00.021978 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1253/2519] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:12:59 2022] PUT /identity/v3/projects/2f78cf71665d4fc5be5038d488422721/users/ee98c450ea03478a9a11d3429ffe143a/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:00.091080 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:00.091365 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:00.091646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:00.119957 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:00.121774 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:00.129595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] MFA Rules not processed for user `ee98c450ea03478a9a11d3429ffe143a`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:00.131067 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:00.199266 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:00.223650 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18380a93-d611-4730-bf0a-f13b7c94bf65 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:00.229383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:00.230027 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:00.230342 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:00.230695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] PATH_INFO: `/v3/projects/4e17c0f6d7a047e68f78b8aac3ee06af` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:00.231308 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1267/2520] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:00.232092 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e17c0f6d7a047e68f78b8aac3ee06af)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:00.234152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:00.240287 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:00.247209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:00.319626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c8e83c-d9e0-44c6-8592-5746b8e87a75 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:00.323254 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:00.323925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:00.324276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:00.324615 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:00.334920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:00.336394 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:00.347681 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1268/2521] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:00.385389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:00.403090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e17c0f6d7a047e68f78b8aac3ee06af', 'request_id': 'req-cd305e32-b912-439a-9907-942a5ce11e81'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.507484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e17c0f6d7a047e68f78b8aac3ee06af', 'request_id': 'req-cd305e32-b912-439a-9907-942a5ce11e81'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.513495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e17c0f6d7a047e68f78b8aac3ee06af', 'request_id': 'req-cd305e32-b912-439a-9907-942a5ce11e81'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.525627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] The token cache is being invalidate because project 4e17c0f6d7a047e68f78b8aac3ee06af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:00.525921 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.526595 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.532962 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1254/2522] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:00 2022] DELETE /identity/v3/projects/4e17c0f6d7a047e68f78b8aac3ee06af => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:00.534660 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:00.545319 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a398c57-a237-476b-95b5-63666fddf33f service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:00.618076 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:00.618678 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:00.619038 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:00.619379 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] PATH_INFO: `/v3/users/57436f95424a45aea99cfc003dff0d50` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:00.623387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57436f95424a45aea99cfc003dff0d50)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:00.623387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:00.812074 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57436f95424a45aea99cfc003dff0d50'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.826773 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57436f95424a45aea99cfc003dff0d50'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.834255 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57436f95424a45aea99cfc003dff0d50'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.835225 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57436f95424a45aea99cfc003dff0d50'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.835870 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57436f95424a45aea99cfc003dff0d50'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:00.922925 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1269/2523] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:13:00 2022] DELETE /identity/v3/users/57436f95424a45aea99cfc003dff0d50 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:01.484322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:01.484322 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:01.497658 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd305e32-b912-439a-9907-942a5ce11e81 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:01.525303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:01.526848 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:01.569874 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:01.570500 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:01.570822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:01.571125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] PATH_INFO: `/v3/projects/07fef7178ab848179b027939e495de68` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:01.572328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] RBAC: Authorizing `identity:delete_project(project_id=07fef7178ab848179b027939e495de68)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:01.574256 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:01.593683 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c12220-64a6-4841-8dea-b7b19d5bce31 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:01.597449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:01.598052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:01.598328 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:01.602281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:01.602281 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:01.627554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:01.661536 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:01.663008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540f0f0e-711e-4f74-869f-086dff8b835e service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:01.671743 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1270/2524] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:01.728483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07fef7178ab848179b027939e495de68', 'request_id': 'req-d56e0268-0d49-4be5-880f-ca54169d465a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:01.798006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07fef7178ab848179b027939e495de68', 'request_id': 'req-d56e0268-0d49-4be5-880f-ca54169d465a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:01.808159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07fef7178ab848179b027939e495de68', 'request_id': 'req-d56e0268-0d49-4be5-880f-ca54169d465a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:01.824809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d4f9-1da6-4af6-b31a-5f40637904a3 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:01.825102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d4f9-1da6-4af6-b31a-5f40637904a3 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:01.825383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9d4f9-1da6-4af6-b31a-5f40637904a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:01.825941 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] The token cache is being invalidate because project 07fef7178ab848179b027939e495de68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:01.826243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:01.826932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:01.846667 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1255/2525] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:01 2022] DELETE /identity/v3/projects/07fef7178ab848179b027939e495de68 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:01.859356 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f8a9d4f9-1da6-4af6-b31a-5f40637904a3 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:01.860833 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f8a9d4f9-1da6-4af6-b31a-5f40637904a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:01.954006 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1271/2526] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:13:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:13:02.124787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:02.126344 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:02.132863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:02.210676 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e0268-0d49-4be5-880f-ca54169d465a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:02.215759 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:02.216375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:02.216659 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:02.216934 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:02.225835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:02.227457 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:02.235569 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1256/2527] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:13:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:13:03.121092 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859ba5-1d46-4d2d-8fac-6c4549753047 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:03.121577 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859ba5-1d46-4d2d-8fac-6c4549753047 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:03.121679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6859ba5-1d46-4d2d-8fac-6c4549753047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:03.141849 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a6859ba5-1d46-4d2d-8fac-6c4549753047 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:03.143843 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a6859ba5-1d46-4d2d-8fac-6c4549753047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:03.151143 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1272/2528] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:03.163378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:03.174940 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8ef8f5-f437-4cae-9b77-dea98bf9178e service swift] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:03.179977 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:03.180568 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:03.180841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:03.181113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:03.189090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:03.191682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:03.199577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1257/2529] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:13:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:13:03.833471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52021608-18d2-4b03-89c8-6a62ee968444 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:03.833763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52021608-18d2-4b03-89c8-6a62ee968444 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:03.834039 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52021608-18d2-4b03-89c8-6a62ee968444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:03.853666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-52021608-18d2-4b03-89c8-6a62ee968444 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:03.854882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-52021608-18d2-4b03-89c8-6a62ee968444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:03.862621 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1273/2530] 10.222.0.60 () {58 vars in 1009 bytes} [Fri Oct 21 15:13:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:03.875361 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:03.886220 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc47140-31d8-4b2f-a2cb-41fdee947bba service swift] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:03.891339 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:03.891932 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:03.892212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:03.892533 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:03.900623 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:03.902178 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:03.910707 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1258/2531] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:13:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:13:04.477510 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:04.478028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:04.478028 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:04.497390 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] MFA Rules not processed for user `c97099704f6b486092571686ff58b3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:04.498511 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:04.506720 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1274/2532] 10.222.0.60 () {58 vars in 967 bytes} [Fri Oct 21 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:04.519164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:04.531201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0a0c0-9dc9-49e6-963e-a35f7b0e2b5b service swift] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:04.537624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd34f518f5eef401da71b7624c9917d44', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:04.538269 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:04.538550 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:04.538859 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:04.548061 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:04.551054 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:04.559916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1259/2533] 10.222.0.60 () {60 vars in 1384 bytes} [Fri Oct 21 15:13:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 21 15:13:06.647486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:06.648890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:06.705862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae2cd4-4c02-4074-8e56-7ef69ba04438 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:06.709726 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:06.710335 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:06.710614 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:06.710917 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:06.712713 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:06.760104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=100 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:06.775097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:06.775097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2b203-8eef-4e7c-9050-c2592fbf7d9b service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:06.782497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1275/2534] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:06 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:07.237052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:07.237052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:07.306922 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d77f82-0534-4b00-9004-0515f307986b service swift] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:07.311751 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:07.312378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:07.312653 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:07.312924 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] PATH_INFO: `/v3/users/b9d4cbdf1de0440a934d97916734338e` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:07.314025 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9d4cbdf1de0440a934d97916734338e)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:07.316002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:07.496809 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9d4cbdf1de0440a934d97916734338e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:07.497398 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9d4cbdf1de0440a934d97916734338e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:07.503467 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:07.510329 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9d4cbdf1de0440a934d97916734338e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:07.608462 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9d4cbdf1de0440a934d97916734338e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:07.609072 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9d4cbdf1de0440a934d97916734338e'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:07.621925 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1260/2535] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:13:07 2022] DELETE /identity/v3/users/b9d4cbdf1de0440a934d97916734338e => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:07.626411 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:07.627272 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:07.627551 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:07.646820 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:07.648058 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:07.706522 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1276/2536] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:07.716194 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:07.722205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:07.779378 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8644b0-e55a-4900-b1cb-9dde3f25dd86 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:07.779828 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:07.780427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:07.780704 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:07.780996 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:07.790259 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:07.791706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:07.802267 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1261/2537] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:08.036809 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:08.037033 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:08.050508 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec8dbb-0acb-41bf-8237-825f8b04ad10 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:08.061604 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:08.062279 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:08.062618 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:08.063068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] PATH_INFO: `/v3/projects/3a5ca098647949a89a1d3d96ed6e5ed4` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:08.065239 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5ca098647949a89a1d3d96ed6e5ed4)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:08.067404 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:08.203882 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5ca098647949a89a1d3d96ed6e5ed4', 'request_id': 'req-76f9508d-cc48-4414-9518-334993de2978'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:08.259275 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5ca098647949a89a1d3d96ed6e5ed4', 'request_id': 'req-76f9508d-cc48-4414-9518-334993de2978'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:08.266243 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5ca098647949a89a1d3d96ed6e5ed4', 'request_id': 'req-76f9508d-cc48-4414-9518-334993de2978'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:08.278011 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] The token cache is being invalidate because project 3a5ca098647949a89a1d3d96ed6e5ed4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:08.278303 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:08.279004 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-76f9508d-cc48-4414-9518-334993de2978 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:08.280595 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1277/2538] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:08 2022] DELETE /identity/v3/projects/3a5ca098647949a89a1d3d96ed6e5ed4 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:10.474023 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:10.475523 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:10.488906 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f342f-c9d4-445e-9a03-81682e75c048 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:10.545113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:10.545741 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:10.546021 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:10.546295 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] PATH_INFO: `/v3/users/3cbb457fed844cfd99b664e694cefccb` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:10.547715 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbb457fed844cfd99b664e694cefccb)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:10.550312 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:10.735708 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbb457fed844cfd99b664e694cefccb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:10.737164 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbb457fed844cfd99b664e694cefccb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:10.749260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbb457fed844cfd99b664e694cefccb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:10.830898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbb457fed844cfd99b664e694cefccb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:10.831543 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbb457fed844cfd99b664e694cefccb'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:10.836912 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:10.846903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1262/2539] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:13:10 2022] DELETE /identity/v3/users/3cbb457fed844cfd99b664e694cefccb => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:10.851364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:10.851893 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:10.852025 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:10.871056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:10.874389 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:10.931593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1278/2540] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:10.941369 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:10.942870 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:11.001967 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005f9d2-4d64-474b-942f-2984f4385f95 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:11.005668 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:11.006275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:11.006555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:11.006858 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:11.015785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:11.017208 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:11.027289 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1263/2541] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:11.381450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:11.392164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:11.397208 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1165b-f531-4fe2-a506-39e012691ebd None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:11.401904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:11.402555 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:11.402853 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:11.403195 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] PATH_INFO: `/v3/projects/37ca014da4334317afdfadf95250f857` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:11.404364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ca014da4334317afdfadf95250f857)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:11.406265 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:11.415240 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:11.540579 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ca014da4334317afdfadf95250f857', 'request_id': 'req-66cd7724-ba0b-4ffc-b235-2d07e72867c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.654149 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ca014da4334317afdfadf95250f857', 'request_id': 'req-66cd7724-ba0b-4ffc-b235-2d07e72867c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.666086 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ca014da4334317afdfadf95250f857', 'request_id': 'req-66cd7724-ba0b-4ffc-b235-2d07e72867c7'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.687727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] The token cache is being invalidate because project 37ca014da4334317afdfadf95250f857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:11.688014 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.688828 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.691437 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1279/2542] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:11 2022] DELETE /identity/v3/projects/37ca014da4334317afdfadf95250f857 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:11.695563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:11.705756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b247adf-b5de-4bf6-9acb-8e05364a450e service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:11.764414 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:11.764968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:11.765251 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:11.765624 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] PATH_INFO: `/v3/users/7911c5df656144a1a87772532ba70119` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:11.766816 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7911c5df656144a1a87772532ba70119)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:11.768838 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:11.954201 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7911c5df656144a1a87772532ba70119'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.955910 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7911c5df656144a1a87772532ba70119'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:11.968243 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7911c5df656144a1a87772532ba70119'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.025176 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7911c5df656144a1a87772532ba70119'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.025773 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7911c5df656144a1a87772532ba70119'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.038852 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1264/2543] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:13:11 2022] DELETE /identity/v3/users/7911c5df656144a1a87772532ba70119 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:13:12.437972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:12.442397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:12.457367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd7724-ba0b-4ffc-b235-2d07e72867c7 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:12.514470 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:12.515188 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:12.515497 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:12.515771 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] PATH_INFO: `/v3/projects/1ef9cb50a1e1489ca3a787883fea95d6` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:12.516895 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ef9cb50a1e1489ca3a787883fea95d6)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:12.518914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:12.636559 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ef9cb50a1e1489ca3a787883fea95d6', 'request_id': 'req-52afeab2-9012-4d8e-a879-3d22a20b79fe'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.704600 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ef9cb50a1e1489ca3a787883fea95d6', 'request_id': 'req-52afeab2-9012-4d8e-a879-3d22a20b79fe'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.711480 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ef9cb50a1e1489ca3a787883fea95d6', 'request_id': 'req-52afeab2-9012-4d8e-a879-3d22a20b79fe'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.722715 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] The token cache is being invalidate because project 1ef9cb50a1e1489ca3a787883fea95d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:12.723007 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.723658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.725145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1280/2544] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:12 2022] DELETE /identity/v3/projects/1ef9cb50a1e1489ca3a787883fea95d6 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:12.729212 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:12.730617 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:12.744712 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88324c38-02f3-4887-9c33-84de5791bc72 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:12.756878 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:12.804011 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:12.804768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:12.805050 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:12.805325 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] PATH_INFO: `/v3/users/ee98c450ea03478a9a11d3429ffe143a` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:12.806530 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee98c450ea03478a9a11d3429ffe143a)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:12.808999 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:12.987356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee98c450ea03478a9a11d3429ffe143a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:12.988056 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee98c450ea03478a9a11d3429ffe143a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.000331 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee98c450ea03478a9a11d3429ffe143a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.065153 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee98c450ea03478a9a11d3429ffe143a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.065793 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-609bab58-1a83-4c5d-9035-c3bfa5e8a523 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee98c450ea03478a9a11d3429ffe143a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.078308 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1265/2545] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:13:12 2022] DELETE /identity/v3/users/ee98c450ea03478a9a11d3429ffe143a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:13.495594 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:13.497685 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:13.513946 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afeab2-9012-4d8e-a879-3d22a20b79fe admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:13.555799 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:13.588430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:13.589056 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:13.589331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:13.589598 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] PATH_INFO: `/v3/projects/2f78cf71665d4fc5be5038d488422721` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:13.590737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f78cf71665d4fc5be5038d488422721)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:13.592616 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:13.694364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f78cf71665d4fc5be5038d488422721', 'request_id': 'req-954295a1-7211-4f05-9a1d-31db9b1aa854'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.750078 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f78cf71665d4fc5be5038d488422721', 'request_id': 'req-954295a1-7211-4f05-9a1d-31db9b1aa854'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.757501 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f78cf71665d4fc5be5038d488422721', 'request_id': 'req-954295a1-7211-4f05-9a1d-31db9b1aa854'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.768748 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] The token cache is being invalidate because project 2f78cf71665d4fc5be5038d488422721 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81565) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:13:13.769032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.769662 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-954295a1-7211-4f05-9a1d-31db9b1aa854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:13:13.771179 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1281/2546] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:13:13 2022] DELETE /identity/v3/projects/2f78cf71665d4fc5be5038d488422721 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:13.778286 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:13.778807 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:13.778920 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:13.799116 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:13.802282 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:13.858678 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1266/2547] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:13.867499 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104174-7690-4972-b363-2dc2429de592 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:13.867885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104174-7690-4972-b363-2dc2429de592 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:13.868154 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9104174-7690-4972-b363-2dc2429de592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:13.889326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-a9104174-7690-4972-b363-2dc2429de592 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:13.890782 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a9104174-7690-4972-b363-2dc2429de592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:13.899297 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1282/2548] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:13.903717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:13.914802 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed06848-f105-4138-ba8a-c7cdab65bb4a None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:13.919494 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:13.920106 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:13.920419 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:13.920681 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] PATH_INFO: `/v3/domains` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:13.921328 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81564) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:13:13.921631 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:13.924086 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:13.936375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1267/2549] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:13:13.941486 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104174-7690-4972-b363-2dc2429de592 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:13.950737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9104174-7690-4972-b363-2dc2429de592 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:13.956628 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:13.957353 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:13.957633 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:13.957902 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:13.958542 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:13.960454 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.040904 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1283/2550] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:13:14.043577 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:14.054392 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d2dea4-7ed8-4b10-859a-af7028e77871 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:14.075792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:14.116733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:14.117471 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.117758 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.118031 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] PATH_INFO: `/v3/users` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.118711 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:14.120626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.200362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1268/2551] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:13:14 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:13:14.201910 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:14.213763 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeca0b4-7bc1-4916-8464-f7e4b587fe30 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:14.218318 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:14.218935 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.219211 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.219477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.220110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:14.222261 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.232704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1284/2552] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:13:14.237121 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:14.247721 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb03601-0182-42ba-8108-573e7a0bc236 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:14.253162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:14.253766 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.254049 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.254323 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.255002 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:14.256591 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bc0c4-1b56-443d-bc13-d64a006b56b0 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.270224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1269/2553] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:13:14.274816 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:14.285299 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e059a25-32f1-4fb0-962c-1f557fb0eb9a admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:14.289991 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:14.290663 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.291019 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.291364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] PATH_INFO: `/v3/projects/e01624214b124e12a495c748e39dda48/users/a2d10cbd26b04ab88cdbd9ca4bb4b901/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.306407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e01624214b124e12a495c748e39dda48, user_id=a2d10cbd26b04ab88cdbd9ca4bb4b901, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:14.309064 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.408257 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1285/2554] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:13:14 2022] PUT /identity/v3/projects/e01624214b124e12a495c748e39dda48/users/a2d10cbd26b04ab88cdbd9ca4bb4b901/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:13:14.413661 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d289b-862a-4bf1-b104-e4d7ca9e7e8d None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.414018 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d289b-862a-4bf1-b104-e4d7ca9e7e8d None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.414379 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d289b-862a-4bf1-b104-e4d7ca9e7e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.434756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ea6d289b-862a-4bf1-b104-e4d7ca9e7e8d None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:14.436042 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ea6d289b-862a-4bf1-b104-e4d7ca9e7e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:14.504012 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1270/2555] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:14.516367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:14.519103 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:14.597446 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9764f2ed-cad3-49b4-ad45-d177f227e5bd admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:14.601862 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:14.602593 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:14.602914 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:14.603189 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:14.611452 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:14.614116 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:14.615608 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:14.625327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1286/2556] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:16.677634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cadf-5fc5-47aa-9af3-41da601d4f59 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:16.678103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cadf-5fc5-47aa-9af3-41da601d4f59 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:16.678103 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590cadf-5fc5-47aa-9af3-41da601d4f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:16.709776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-5590cadf-5fc5-47aa-9af3-41da601d4f59 None None] MFA Rules not processed for user `a2d10cbd26b04ab88cdbd9ca4bb4b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:16.711044 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-5590cadf-5fc5-47aa-9af3-41da601d4f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:16.791700 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1271/2557] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 21 15:13:16.802954 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:13:16.805770 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:16.873276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3c369-2971-4f43-9936-e47eae3b8b53 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:13:16.878725 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:13:16.879345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:16.879624 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:16.879896 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:16.888956 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:13:16.890447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:13:16.901815 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1287/2558] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:13:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:13:18.645986 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:13:18.646502 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:13:18.646612 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:13:18.665249 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] MFA Rules not processed for user `a2d10cbd26b04ab88cdbd9ca4bb4b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:13:18.666447 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:13:18.674066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1272/2559] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:13:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:13:26.896090 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=28 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:13:28.659850 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=86,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:14:14.257431 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:14:14.262258 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:14.270222 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc323e-dd99-4fb3-87b8-1de709eb9d89 service cinder] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:14:14.273789 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '33932a27fdc0413fac63db9898ef8475', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:14:14.274424 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:14:14.274732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:14:14.275002 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:14:14.276642 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:14.360727 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:14:14.362181 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a1ee9-6bee-4f36-b489-0b24b80037ba service cinder] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:14:14.381972 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1288/2560] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:14:17.388113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:14:17.389509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:17.401258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e7b9-6577-4d2f-b800-5d61e0055211 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:14:17.406000 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:14:17.406604 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:14:17.406911 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:14:17.407188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:14:17.408844 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:17.422785 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:14:17.424449 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:14:17.435203 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1273/2561] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:14:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:14:24.379409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=43 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:14:27.430349 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:14:30.057263 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:14:30.057753 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:14:30.057877 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:14:30.077326 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] MFA Rules not processed for user `a2d10cbd26b04ab88cdbd9ca4bb4b901`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:14:30.079291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:30.086104 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1289/2562] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:14:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:14:40.071981 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:14:47.633363 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:14:47.635148 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:47.651403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb072555-e989-4204-9448-72b55b05a4f8 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:14:47.657768 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:14:47.658804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:14:47.658804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:14:47.659030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:14:47.661466 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:47.675488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:14:47.677356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:14:47.689688 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1274/2563] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:14:57.684417 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:14:59.599739 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:14:59.609688 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f24f10-52ff-4507-912f-45753ba0480f None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:14:59.613717 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:14:59.614322 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:14:59.614592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:14:59.614890 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:14:59.616586 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:14:59.626416 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:14:59.627885 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:14:59.636930 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1290/2564] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:15:05.565756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:05.576090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bb41f4-52c2-41ac-9518-33d95bf5f339 service nova] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:05.580376 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:05.580987 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:05.581268 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:05.581542 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:05.583512 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:05.595717 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:05.597197 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:05.607488 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1275/2565] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:15:09.633874 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=6 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:13.402271 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:13.403905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:13.419530 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d35bb4-cc14-45db-ae14-eda8ee4120b6 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:13.423951 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:13.424824 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:13.425095 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:13.425357 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] PATH_INFO: `/v3/users/f71ffc3d05d4470fa533cfd6f22184b2` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:13.426872 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71ffc3d05d4470fa533cfd6f22184b2)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:13.428427 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:13.604644 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71ffc3d05d4470fa533cfd6f22184b2'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:13.617045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71ffc3d05d4470fa533cfd6f22184b2'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:13.617776 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71ffc3d05d4470fa533cfd6f22184b2'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:13.625052 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71ffc3d05d4470fa533cfd6f22184b2'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:13.625684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71ffc3d05d4470fa533cfd6f22184b2'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:13.686887 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1291/2566] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:15:13 2022] DELETE /identity/v3/users/f71ffc3d05d4470fa533cfd6f22184b2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:13.922756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:13.935066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263f7d1-73ec-4d93-a8ea-784de2ed0f83 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:13.996099 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:13.996738 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:13.997030 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:13.997291 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] PATH_INFO: `/v3/projects/57eb0443c72743c8b3ab016fdf5c9745` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:13.998401 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] RBAC: Authorizing `identity:delete_project(project_id=57eb0443c72743c8b3ab016fdf5c9745)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:14.000332 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:14.119804 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57eb0443c72743c8b3ab016fdf5c9745', 'request_id': 'req-f656a23c-d9ce-4f07-aa47-ac58d09f893a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.182402 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57eb0443c72743c8b3ab016fdf5c9745', 'request_id': 'req-f656a23c-d9ce-4f07-aa47-ac58d09f893a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.188638 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57eb0443c72743c8b3ab016fdf5c9745', 'request_id': 'req-f656a23c-d9ce-4f07-aa47-ac58d09f893a'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.200465 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] The token cache is being invalidate because project 57eb0443c72743c8b3ab016fdf5c9745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:15:14.200792 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.201509 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.203013 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1276/2567] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:15:13 2022] DELETE /identity/v3/projects/57eb0443c72743c8b3ab016fdf5c9745 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:14.207164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:14.209192 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:14.222363 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3f53d-a0b8-47bc-a289-10e89cc67f6d admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:14.277704 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:14.278332 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:14.278607 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:14.278905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] PATH_INFO: `/v3/users/6e3aafdb629044f78c2b2e289e5409d3` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:14.280032 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e3aafdb629044f78c2b2e289e5409d3)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:14.281941 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:14.438229 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e3aafdb629044f78c2b2e289e5409d3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.450479 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e3aafdb629044f78c2b2e289e5409d3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.451228 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e3aafdb629044f78c2b2e289e5409d3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.458755 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e3aafdb629044f78c2b2e289e5409d3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.459430 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e3aafdb629044f78c2b2e289e5409d3'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:14.461094 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:14.538343 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1292/2568] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:15:14 2022] DELETE /identity/v3/users/6e3aafdb629044f78c2b2e289e5409d3 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:14.767615 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:14.769158 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:14.784723 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656a23c-d9ce-4f07-aa47-ac58d09f893a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:14.846837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:14.847473 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:14.847756 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:14.848029 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] PATH_INFO: `/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:14.849187 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cb6b4f5f9aa4202a7104f39187d5766)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:14.851179 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:14.872907 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:14.964255 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cb6b4f5f9aa4202a7104f39187d5766', 'request_id': 'req-981118a6-c596-4c7f-808c-cea06b8679ea'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:15.058066 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cb6b4f5f9aa4202a7104f39187d5766', 'request_id': 'req-981118a6-c596-4c7f-808c-cea06b8679ea'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:15.063984 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cb6b4f5f9aa4202a7104f39187d5766', 'request_id': 'req-981118a6-c596-4c7f-808c-cea06b8679ea'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:15.075991 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] The token cache is being invalidate because project 4cb6b4f5f9aa4202a7104f39187d5766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:15:15.076275 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:15.076913 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:15.078393 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1277/2569] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:15:14 2022] DELETE /identity/v3/projects/4cb6b4f5f9aa4202a7104f39187d5766 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:24.595712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:25.080871 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:35.008477 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:35.009962 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:35.023512 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a59bb-16ca-4332-97d1-0b9d36c51075 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:35.082449 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:35.083137 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:35.083412 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:35.083684 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] PATH_INFO: `/v3/users/a2d10cbd26b04ab88cdbd9ca4bb4b901` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:35.084865 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d10cbd26b04ab88cdbd9ca4bb4b901)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:35.086839 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:35.287045 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d10cbd26b04ab88cdbd9ca4bb4b901'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.299426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d10cbd26b04ab88cdbd9ca4bb4b901'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.300351 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d10cbd26b04ab88cdbd9ca4bb4b901'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.307268 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d10cbd26b04ab88cdbd9ca4bb4b901'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.307891 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-1ef0dbd7-2a4f-432c-ba4b-6072d8170b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d10cbd26b04ab88cdbd9ca4bb4b901'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.378592 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1293/2570] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:15:35 2022] DELETE /identity/v3/users/a2d10cbd26b04ab88cdbd9ca4bb4b901 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:35.664686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:35.666117 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:35.680337 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981118a6-c596-4c7f-808c-cea06b8679ea admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:35.738306 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:35.739126 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:35.739627 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:35.740010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] PATH_INFO: `/v3/projects/e01624214b124e12a495c748e39dda48` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:35.741260 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e01624214b124e12a495c748e39dda48)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:35.743283 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:35.835006 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e01624214b124e12a495c748e39dda48', 'request_id': 'req-0b2c2789-6edf-4b18-a55c-5a84efda8634'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.892016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e01624214b124e12a495c748e39dda48', 'request_id': 'req-0b2c2789-6edf-4b18-a55c-5a84efda8634'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.897950 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e01624214b124e12a495c748e39dda48', 'request_id': 'req-0b2c2789-6edf-4b18-a55c-5a84efda8634'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.910016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] The token cache is being invalidate because project e01624214b124e12a495c748e39dda48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:15:35.910296 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.910958 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-0b2c2789-6edf-4b18-a55c-5a84efda8634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:15:35.912498 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1278/2571] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:15:35 2022] DELETE /identity/v3/projects/e01624214b124e12a495c748e39dda48 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:35.917905 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:35.918191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:35.918468 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:35.939380 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:35.943407 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:36.003276 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1294/2572] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:36.011602 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.011893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.012171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.034043 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:36.035284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:36.043188 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1279/2573] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:36.047561 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.057574 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d387bc-df23-41f3-957f-08204a44daf8 None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.062125 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.063008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.063286 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.063554 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] PATH_INFO: `/v3/domains` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.064191 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81565) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 21 15:15:36.064488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.066068 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.077393 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1295/2574] 10.222.0.60 () {58 vars in 1153 bytes} [Fri Oct 21 15:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 21 15:15:36.082195 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.092866 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7c6ad-de10-4933-8e6c-eb7d3dacbba1 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.096963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.097555 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.097835 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.098108 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.098776 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.100656 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.167863 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1280/2575] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:15:36 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:15:36.172175 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.182488 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b2734-a434-453b-a6ee-b1e4e45d1439 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.185995 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:36.239153 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.239757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.240101 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.240563 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.241340 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.243732 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.309986 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1296/2576] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:15:36 2022] POST /identity/v3/users => generated 286 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:15:36.316134 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.326868 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd18e7-1c9f-4bcb-9c14-e29aa7eedb0c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.332501 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.333096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.333375 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.333648 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.334284 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.335914 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.345893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1281/2577] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:15:36.349612 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.359565 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ea406-f70a-43b0-b819-2681c669ca16 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.364737 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.367309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.367309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.367309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.367309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.368209 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08db17f7-d660-49df-8018-3be1ffa99866 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.379521 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1297/2578] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:15:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:15:36.383525 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.394403 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17104073-554b-4ab4-9ac5-95c63c2fab6e admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.399136 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.399803 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] REQUEST_METHOD: `PUT` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.400083 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.400355 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] PATH_INFO: `/v3/projects/791930b581eb4c55bebd1e6eacc6b4c0/users/2eee332e02954cf0a0478a4b98b65f86/roles/8a04c4ea718349e293dc17454cef7b86` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.415258 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=791930b581eb4c55bebd1e6eacc6b4c0, user_id=2eee332e02954cf0a0478a4b98b65f86, role_id=8a04c4ea718349e293dc17454cef7b86)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.417899 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.480634 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1282/2579] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:15:36 2022] PUT /identity/v3/projects/791930b581eb4c55bebd1e6eacc6b4c0/users/2eee332e02954cf0a0478a4b98b65f86/roles/8a04c4ea718349e293dc17454cef7b86 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:36.486161 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324261b-7285-4e6c-b074-85034d4774b4 None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.486447 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324261b-7285-4e6c-b074-85034d4774b4 None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.486749 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324261b-7285-4e6c-b074-85034d4774b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.506270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-0324261b-7285-4e6c-b074-85034d4774b4 None None] MFA Rules not processed for user `332151c44437481cb5ef58e0902276ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:36.507491 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-0324261b-7285-4e6c-b074-85034d4774b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:36.566901 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1298/2580] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:36.576404 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:36.579214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:36.599564 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:36.635663 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0c33-2f93-451f-a156-5a9207a4e22a admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:36.639356 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:36.639955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:36.642747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:36.642747 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:36.649862 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:36.651305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:36.661695 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1283/2581] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:15:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:15:41.968213 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:41.968783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:41.968783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.006712 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.auth.core [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] MFA Rules not processed for user `2eee332e02954cf0a0478a4b98b65f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81565) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:42.007928 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:42.077397 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1299/2582] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:42.082929 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.092864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc01f7-48f3-424f-886e-a1ffaede7077 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.097671 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.098281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.098565 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.098926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.099596 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.101489 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.236324 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1284/2583] 10.222.0.60 () {60 vars in 1153 bytes} [Fri Oct 21 15:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 21 15:15:42.240538 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.252599 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca8b2a9-334b-4f32-854c-b902bbee5d9e None None] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.264695 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:15:42.311210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.311945 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] REQUEST_METHOD: `POST` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.312805 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.313462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] PATH_INFO: `/v3/users` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.313724 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.315783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.430462 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1300/2584] 10.222.0.60 () {60 vars in 1144 bytes} [Fri Oct 21 15:15:42 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 21 15:15:42.434380 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.444806 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dba486-f04d-4ed0-8ba1-c25c5307bc0c admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.449348 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.449938 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.450213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.450483 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.451170 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.452746 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.463632 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1285/2585] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:15:42.467777 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.478505 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0dd8aa-3b4d-46b5-b8d8-4c0ef2520128 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.483207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.483858 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.484783 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.485061 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] PATH_INFO: `/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a/users/654d8f90f67f44fc905febae5529cd32/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.500164 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ea7a8d51c94095bf9cda1c73562c6a, user_id=654d8f90f67f44fc905febae5529cd32, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.502850 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.573046 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1301/2586] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:15:42 2022] PUT /identity/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a/users/654d8f90f67f44fc905febae5529cd32/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:42.577010 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.587561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb19dc4-46e6-44d3-a344-97f7626e7011 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.644428 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.645028 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.645304 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.645573 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] PATH_INFO: `/v3/roles` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.646198 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.647903 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941ee1a-fcda-4294-a0cd-9ce1090b584f admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.658096 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1286/2587] 10.222.0.60 () {58 vars in 1122 bytes} [Fri Oct 21 15:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 21 15:15:42.662345 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:42.673212 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda9d-4260-4ca7-9317-3db7350046e9 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:42.678145 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:42.678829 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:42.679111 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:42.679388 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] PATH_INFO: `/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a/users/654d8f90f67f44fc905febae5529cd32/roles/55a557362b9542629b4209da47492ebf` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:42.681331 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ea7a8d51c94095bf9cda1c73562c6a, user_id=654d8f90f67f44fc905febae5529cd32, role_id=55a557362b9542629b4209da47492ebf)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:42.683975 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:42.694043 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1302/2588] 10.222.0.60 () {60 vars in 1483 bytes} [Fri Oct 21 15:15:42 2022] PUT /identity/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a/users/654d8f90f67f44fc905febae5529cd32/roles/55a557362b9542629b4209da47492ebf => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:15:48.384107 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb31-813f-42ef-8299-6bfb325dfac2 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:48.384613 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb31-813f-42ef-8299-6bfb325dfac2 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:48.384720 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb31-813f-42ef-8299-6bfb325dfac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:48.416409 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-3e4ccb31-813f-42ef-8299-6bfb325dfac2 None None] MFA Rules not processed for user `654d8f90f67f44fc905febae5529cd32`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:48.417636 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-3e4ccb31-813f-42ef-8299-6bfb325dfac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:48.489945 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1287/2589] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:48.505330 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:15:48.507127 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:48.582260 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3371ed-1c9c-4709-a85b-1c4249b975b2 admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:15:48.587765 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:15:48.588348 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:48.588626 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:48.588899 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:48.644766 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:15:48.646355 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:15:48.655366 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1303/2590] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:15:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:15:50.952224 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] REQUEST_METHOD: `POST` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:15:50.954476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:15:50.954837 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:15:50.977864 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.auth.core [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] MFA Rules not processed for user `2eee332e02954cf0a0478a4b98b65f86`. Rule list: `[]` (Enabled: `True`). {{(pid=81564) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 21 15:15:50.979129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:15:50.986955 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1288/2591] 10.222.0.60 () {58 vars in 958 bytes} [Fri Oct 21 15:15:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 21 15:15:58.651215 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:16:00.972015 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:16:06.892073 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:16:06.946327 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e113c88f-8e61-4278-97a4-56f7140543bb service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:16:06.953050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:16:06.953050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:16:06.953050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:16:06.953050 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:16:06.953463 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:06.962988 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:16:06.964409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:16:06.972785 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1304/2592] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:16:07.650841 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:16:07.652330 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:07.724090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35762015-9638-42f9-aabd-3256f38d25b3 None None] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:16:07.729073 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:16:07.729682 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:16:07.729963 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:16:07.730237 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:16:07.731968 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:07.743552 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:16:07.745119 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:16:07.755603 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1289/2593] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:16:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:16:16.968666 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=26 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:16:17.750649 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=32 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:16:23.057346 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:16:23.058826 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:23.075461 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630fbc73-f0e6-446a-ae35-0447de5e8cfe service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:16:23.082367 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:16:23.082721 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:16:23.083185 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:16:23.083464 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:16:23.085171 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:23.097445 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:16:23.098920 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:16:23.109706 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1305/2594] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:16:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:16:25.337218 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:16:25.346608 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd11b30-c4b5-49fb-9d9e-27a9e7f515d5 service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:16:25.350890 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:16:25.351476 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:16:25.351754 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:16:25.352022 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:16:25.360822 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:16:25.362299 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:16:25.372981 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1290/2595] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:16:27.520602 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:16:27.532036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f88668-9428-416f-b194-7d1635b996c9 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:16:27.538466 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:16:27.539102 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:16:27.539432 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:16:27.539658 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:16:27.541387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:16:27.555110 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:16:27.556710 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:16:27.567566 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1306/2596] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:16:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:16:35.368484 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:16:37.563008 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=12 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:17:26.250171 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:17:26.255626 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:17:26.266047 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679de538-7713-4abc-97b6-6466666d1bb6 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:17:26.271129 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:17:26.271733 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:17:26.272016 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:17:26.272290 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:17:26.274053 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:17:26.286522 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:17:26.288125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:17:26.298040 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1291/2597] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:17:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:17:36.293737 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:18:37.183646 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:18:37.193077 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b65f3-550a-465f-aeaf-4be550f02f92 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:18:37.196757 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:18:37.197293 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:18:37.197571 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:18:37.198506 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:18:37.200270 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:18:37.209637 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:18:37.211088 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:18:37.218963 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1307/2598] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:18:37.570113 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:18:37.571641 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:18:37.584069 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9dc1f9-4502-42a0-9245-ef43273d4e75 service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:18:37.587677 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:18:37.590305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:18:37.590305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:18:37.590305 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:18:37.591353 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:18:37.601451 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:18:37.602898 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:18:37.612125 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1292/2599] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:18:37.698487 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:18:37.706924 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30344f05-f0a3-44c4-b0a2-2d9fa6209779 service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:18:37.710700 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:18:37.711364 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:18:37.711647 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:18:37.711919 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:18:37.714373 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:18:37.727545 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:18:37.729697 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:18:37.740509 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1308/2600] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:18:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:18:47.608038 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=6 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:18:47.735699 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=12 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:20:09.737504 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:09.747472 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8385f-bb15-4c3d-9de8-a2f49ee24cfa service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:09.752314 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:09.752926 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:09.753202 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:09.753495 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:09.756213 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:09.767812 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:09.769526 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:09.778425 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1293/2601] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:20:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:20:10.089221 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:10.098779 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ecfc97-d9c7-4cbf-b046-dd1c6808c89a service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:10.102868 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:10.103503 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:10.103786 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:10.104059 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:10.105813 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:10.121731 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:10.123210 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:10.135242 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1309/2602] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:20:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:20:11.331725 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:11.341233 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5f4f5-2efc-48b4-b778-099a5e641766 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:11.344893 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce92d16f3ee44e9eb2a65978edc3938e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:11.345633 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:11.345916 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:11.346281 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:11.348060 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:11.362546 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:11.364003 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:11.375217 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1294/2603] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:20:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 21 15:20:12.075282 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:12.087036 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6aec2-efd0-4df1-be95-d66ab68d3fce service neutron] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:12.092547 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee990610dd642059936212305d94fb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:12.093135 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:12.093409 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:12.093679 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:12.096309 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:12.108182 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:12.109729 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:12.122224 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1310/2604] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:20:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:20:17.393175 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:17.404753 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a888b-ec28-4957-8f5a-cc6bbc303ac9 service neutron] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:17.410438 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:17.411068 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] REQUEST_METHOD: `GET` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:17.411427 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:17.411750 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:17.413468 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:17.426090 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:17.427692 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:17.438156 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1295/2605] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:20:21.223982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:21.287207 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59ae451-3197-491a-9978-6608a2f8c782 service nova] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:21.291761 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:21.292387 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:21.292664 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:21.292936 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] PATH_INFO: `/v3/users/2eee332e02954cf0a0478a4b98b65f86` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:21.294057 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eee332e02954cf0a0478a4b98b65f86)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:21.296018 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:21.516611 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eee332e02954cf0a0478a4b98b65f86'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:21.529230 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eee332e02954cf0a0478a4b98b65f86'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:21.531097 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eee332e02954cf0a0478a4b98b65f86'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:21.538072 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eee332e02954cf0a0478a4b98b65f86'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:21.538759 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eee332e02954cf0a0478a4b98b65f86'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:21.605063 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1311/2606] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:20:21 2022] DELETE /identity/v3/users/2eee332e02954cf0a0478a4b98b65f86 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:20:21.859960 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:21.870152 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57919c39-52b3-422e-b85c-d630622b263d service placement] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:21.928686 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:21.929561 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:21.929834 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:21.930104 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] PATH_INFO: `/v3/projects/791930b581eb4c55bebd1e6eacc6b4c0` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:21.931227 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=791930b581eb4c55bebd1e6eacc6b4c0)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:21.933097 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:22.052311 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '791930b581eb4c55bebd1e6eacc6b4c0', 'request_id': 'req-c961b393-a6de-486a-9aae-5359a6f0ac65'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.131300 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '791930b581eb4c55bebd1e6eacc6b4c0', 'request_id': 'req-c961b393-a6de-486a-9aae-5359a6f0ac65'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.137205 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '791930b581eb4c55bebd1e6eacc6b4c0', 'request_id': 'req-c961b393-a6de-486a-9aae-5359a6f0ac65'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.150391 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] The token cache is being invalidate because project 791930b581eb4c55bebd1e6eacc6b4c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:20:22.150706 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.151362 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.152882 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1296/2607] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:20:21 2022] DELETE /identity/v3/projects/791930b581eb4c55bebd1e6eacc6b4c0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 21 15:20:22.157087 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:22.158526 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:22.171821 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193e7954-66b5-4d62-95ea-d6e53e99f5cd admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:22.220866 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:20:22.229524 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:22.237906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:22.237906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:22.237906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] PATH_INFO: `/v3/users/654d8f90f67f44fc905febae5529cd32` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:22.237906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=654d8f90f67f44fc905febae5529cd32)` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:22.237906 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:22.407819 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654d8f90f67f44fc905febae5529cd32'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.419982 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654d8f90f67f44fc905febae5529cd32'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.420650 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654d8f90f67f44fc905febae5529cd32'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.428383 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654d8f90f67f44fc905febae5529cd32'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.429030 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.notifications [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654d8f90f67f44fc905febae5529cd32'} {{(pid=81565) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:22.503113 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1312/2608] 10.222.0.60 () {58 vars in 1224 bytes} [Fri Oct 21 15:20:22 2022] DELETE /identity/v3/users/654d8f90f67f44fc905febae5529cd32 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:20:22.737997 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Authenticating user token {{(pid=81564) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:20:22.739454 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.fernet_utils [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81564) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:20:22.754431 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961b393-a6de-486a-9aae-5359a6f0ac65 admin admin] Validating token access rules against request {{(pid=81564) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:20:22.803800 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:20:22.813897 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '332151c44437481cb5ef58e0902276ac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1cd93ea5b05642848d07178b50fbac9f', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81564) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:20:22.814563 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:20:22.814886 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:20:22.815162 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] PATH_INFO: `/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a` {{(pid=81564) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:20:22.816288 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ea7a8d51c94095bf9cda1c73562c6a)` {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:20:22.818214 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] RBAC: Authorization granted {{(pid=81564) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:20:22.924787 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ea7a8d51c94095bf9cda1c73562c6a', 'request_id': 'req-4ac11ef6-c9ca-495a-8a14-3017a37e6115'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:23.016159 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ea7a8d51c94095bf9cda1c73562c6a', 'request_id': 'req-4ac11ef6-c9ca-495a-8a14-3017a37e6115'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:23.022943 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ea7a8d51c94095bf9cda1c73562c6a', 'request_id': 'req-4ac11ef6-c9ca-495a-8a14-3017a37e6115'} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:23.034799 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] The token cache is being invalidate because project 81ea7a8d51c94095bf9cda1c73562c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81564) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 21 15:20:23.037383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:23.037383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG keystone.notifications [None req-4ac11ef6-c9ca-495a-8a14-3017a37e6115 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81564) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 21 15:20:23.037383 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: [pid: 81564|app: 0|req: 1297/2609] 10.222.0.60 () {58 vars in 1233 bytes} [Fri Oct 21 15:20:22 2022] DELETE /identity/v3/projects/81ea7a8d51c94095bf9cda1c73562c6a => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 21 15:20:32.499133 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:20:33.031052 dv-cinder-836083-16-iscsi devstack@keystone.service[81564]: DEBUG dbcounter [-] [81564] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81564) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 21 15:22:32.226471 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Authenticating user token {{(pid=81565) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 21 15:22:32.227911 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:22:32.370426 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6c4f1-595f-431b-8deb-8ec4d5ab46fe admin admin] Validating token access rules against request {{(pid=81565) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 21 15:22:32.429134 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e05f107798a24fa9a0e65918076f2cc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd59dd3ce7504494bb481758b4195d883', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81565) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 21 15:22:32.429741 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] REQUEST_METHOD: `GET` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 21 15:22:32.430020 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] SCRIPT_NAME: `/identity` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 21 15:22:32.430291 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81565) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 21 15:22:32.432280 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.fernet_utils [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81565) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 21 15:22:32.443395 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 21 15:22:32.444984 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d26746-2c5e-45e3-8f8d-84253d014340 service placement] RBAC: Authorization granted {{(pid=81565) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 21 15:22:32.454744 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: [pid: 81565|app: 0|req: 1313/2610] 10.222.0.60 () {60 vars in 1365 bytes} [Fri Oct 21 15:22:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 21 15:22:42.450450 dv-cinder-836083-16-iscsi devstack@keystone.service[81565]: DEBUG dbcounter [-] [81565] Writing DB stats keystone:SELECT=89 {{(pid=81565) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}